{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 零基础实战机器学习\n",
    "\n",
    "## 第17讲 集成学习-裂变转化预测\n",
    "\n",
    "作者 黄佳\n",
    "\n",
    "极客时间专栏链接：https://time.geekbang.org/column/intro/438\n",
    "\n",
    "\n",
    "问题：通过集成学习方法预测用户是否会购买促销品\n",
    "\n",
    "易速鲜花公司推出两张促销方案，分别是疯狂打折和买一送一。\n",
    "\n",
    "通过集成学习方法，我们可以预测出用户是否会购买这些打折商品。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "用户数: 10000\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>用户码</th>\n",
       "      <th>性别</th>\n",
       "      <th>年龄</th>\n",
       "      <th>城市</th>\n",
       "      <th>入会月数</th>\n",
       "      <th>会员类型</th>\n",
       "      <th>活跃用户</th>\n",
       "      <th>会员积分</th>\n",
       "      <th>总消费</th>\n",
       "      <th>年收入</th>\n",
       "      <th>裂变方案</th>\n",
       "      <th>是否转化</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>女</td>\n",
       "      <td>40</td>\n",
       "      <td>杭州</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>516</td>\n",
       "      <td>6360.66</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>疯狂打折</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>男</td>\n",
       "      <td>69</td>\n",
       "      <td>杭州</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>682</td>\n",
       "      <td>28605.00</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>疯狂打折</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>女</td>\n",
       "      <td>32</td>\n",
       "      <td>杭州</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>803</td>\n",
       "      <td>10378.09</td>\n",
       "      <td>236311.0932</td>\n",
       "      <td>买一送一</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>女</td>\n",
       "      <td>37</td>\n",
       "      <td>苏州</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>778</td>\n",
       "      <td>25564.01</td>\n",
       "      <td>129909.8079</td>\n",
       "      <td>买一送一</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>男</td>\n",
       "      <td>55</td>\n",
       "      <td>苏州</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>547</td>\n",
       "      <td>3235.61</td>\n",
       "      <td>136976.1948</td>\n",
       "      <td>买一送一</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>9996</td>\n",
       "      <td>男</td>\n",
       "      <td>49</td>\n",
       "      <td>扬州</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>746</td>\n",
       "      <td>1930.44</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>疯狂打折</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>9997</td>\n",
       "      <td>男</td>\n",
       "      <td>36</td>\n",
       "      <td>杭州</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>569</td>\n",
       "      <td>29593.82</td>\n",
       "      <td>82517.8587</td>\n",
       "      <td>买一送一</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>9998</td>\n",
       "      <td>女</td>\n",
       "      <td>55</td>\n",
       "      <td>扬州</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>850</td>\n",
       "      <td>32841.84</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>疯狂打折</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>9999</td>\n",
       "      <td>男</td>\n",
       "      <td>35</td>\n",
       "      <td>扬州</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>529</td>\n",
       "      <td>21564.34</td>\n",
       "      <td>117800.7531</td>\n",
       "      <td>买一送一</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10000</td>\n",
       "      <td>女</td>\n",
       "      <td>48</td>\n",
       "      <td>苏州</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>657</td>\n",
       "      <td>19371.82</td>\n",
       "      <td>176622.9201</td>\n",
       "      <td>疯狂打折</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        用户码 性别  年龄  城市  入会月数  会员类型  活跃用户  会员积分       总消费          年收入  裂变方案  \\\n",
       "0         1  女  40  杭州     9     2     1   516   6360.66       0.0000  疯狂打折   \n",
       "1         2  男  69  杭州     6     2     1   682  28605.00       0.0000  疯狂打折   \n",
       "2         3  女  32  杭州     9     1     1   803  10378.09  236311.0932  买一送一   \n",
       "3         4  女  37  苏州     0     2     1   778  25564.01  129909.8079  买一送一   \n",
       "4         5  男  55  苏州     4     3     0   547   3235.61  136976.1948  买一送一   \n",
       "...     ... ..  ..  ..   ...   ...   ...   ...       ...          ...   ...   \n",
       "9995   9996  男  49  扬州     7     2     1   746   1930.44       0.0000  疯狂打折   \n",
       "9996   9997  男  36  杭州     1     1     0   569  29593.82   82517.8587  买一送一   \n",
       "9997   9998  女  55  扬州     7     1     0   850  32841.84       0.0000  疯狂打折   \n",
       "9998   9999  男  35  扬州     5     1     1   529  21564.34  117800.7531  买一送一   \n",
       "9999  10000  女  48  苏州     5     1     0   657  19371.82  176622.9201  疯狂打折   \n",
       "\n",
       "      是否转化  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  \n",
       "...    ...  \n",
       "9995     0  \n",
       "9996     0  \n",
       "9997     1  \n",
       "9998     0  \n",
       "9999     1  \n",
       "\n",
       "[10000 rows x 12 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd #导入Pandas\n",
    "import numpy as np #导入NumPy\n",
    "df_fission = pd.read_csv('易速鲜花裂变转化.csv') #载入数据\n",
    "print('用户数:', df_fission.count()['用户码']) #查看数据条目数\n",
    "df_fission #显示头几行数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gender unique values [0 1]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>用户码</th>\n",
       "      <th>性别</th>\n",
       "      <th>年龄</th>\n",
       "      <th>入会月数</th>\n",
       "      <th>会员类型</th>\n",
       "      <th>活跃用户</th>\n",
       "      <th>会员积分</th>\n",
       "      <th>总消费</th>\n",
       "      <th>年收入</th>\n",
       "      <th>是否转化</th>\n",
       "      <th>城市_杭州</th>\n",
       "      <th>城市_苏州</th>\n",
       "      <th>裂变方案_疯狂打折</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>516</td>\n",
       "      <td>6360.66</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>69</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>682</td>\n",
       "      <td>28605.00</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>803</td>\n",
       "      <td>10378.09</td>\n",
       "      <td>236311.0932</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>778</td>\n",
       "      <td>25564.01</td>\n",
       "      <td>129909.8079</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>547</td>\n",
       "      <td>3235.61</td>\n",
       "      <td>136976.1948</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>9996</td>\n",
       "      <td>1</td>\n",
       "      <td>49</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>746</td>\n",
       "      <td>1930.44</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>9997</td>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>569</td>\n",
       "      <td>29593.82</td>\n",
       "      <td>82517.8587</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>9998</td>\n",
       "      <td>0</td>\n",
       "      <td>55</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>850</td>\n",
       "      <td>32841.84</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>9999</td>\n",
       "      <td>1</td>\n",
       "      <td>35</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>529</td>\n",
       "      <td>21564.34</td>\n",
       "      <td>117800.7531</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10000</td>\n",
       "      <td>0</td>\n",
       "      <td>48</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>657</td>\n",
       "      <td>19371.82</td>\n",
       "      <td>176622.9201</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        用户码  性别  年龄  入会月数  会员类型  活跃用户  会员积分       总消费          年收入  是否转化  \\\n",
       "0         1   0  40     9     2     1   516   6360.66       0.0000     0   \n",
       "1         2   1  69     6     2     1   682  28605.00       0.0000     0   \n",
       "2         3   0  32     9     1     1   803  10378.09  236311.0932     1   \n",
       "3         4   0  37     0     2     1   778  25564.01  129909.8079     1   \n",
       "4         5   1  55     4     3     0   547   3235.61  136976.1948     1   \n",
       "...     ...  ..  ..   ...   ...   ...   ...       ...          ...   ...   \n",
       "9995   9996   1  49     7     2     1   746   1930.44       0.0000     0   \n",
       "9996   9997   1  36     1     1     0   569  29593.82   82517.8587     0   \n",
       "9997   9998   0  55     7     1     0   850  32841.84       0.0000     1   \n",
       "9998   9999   1  35     5     1     1   529  21564.34  117800.7531     0   \n",
       "9999  10000   0  48     5     1     0   657  19371.82  176622.9201     1   \n",
       "\n",
       "      城市_杭州  城市_苏州  裂变方案_疯狂打折  \n",
       "0         1      0          1  \n",
       "1         1      0          1  \n",
       "2         1      0          0  \n",
       "3         0      1          0  \n",
       "4         0      1          0  \n",
       "...     ...    ...        ...  \n",
       "9995      0      0          1  \n",
       "9996      1      0          0  \n",
       "9997      0      0          1  \n",
       "9998      0      0          0  \n",
       "9999      0      1          1  \n",
       "\n",
       "[10000 rows x 13 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 把二元类别文本数字化\n",
    "df_fission['性别'].replace(\"女\",0,inplace = True)\n",
    "df_fission['性别'].replace(\"男\",1,inplace=True)\n",
    "# 显示数字类别\n",
    "print(\"Gender unique values\",df_fission['性别'].unique())\n",
    "# 把多元类别转换成多个二元哑变量，然后贴回原始数据集\n",
    "df_fission = pd.get_dummies(df_fission, drop_first = True)\n",
    "# df_fission = [df_fission, d_city]\n",
    "# df_fission = pd.concat(df_bank, axis = 1)\n",
    "# 构建特征和标签集合\n",
    "# y = df_fission['Exited']\n",
    "# X = df_fission.drop(['Name', 'Exited', 'City'], axis=1)\n",
    "# X.head() #显示新的特征集\n",
    "df_fission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\jacky.huang\\Anaconda3\\lib\\site-packages\\seaborn\\_decorators.py:36: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEECAYAAADHzyg1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQc0lEQVR4nO3de4xcZ33G8e8THHC4bWxnUyLTJGpRUUvAFSwUQWixqQ0UAyWUNgXTEFpMEZRW3KRU6pVSAaEVqEUBq4SCZSFRAgVcBRJsI0i4hA1g03JpuQQwArrGxW4KMZf8+sc5frU7cbJjyJndZL8fabTv+b0zZ9+RVufZ9z1zzqSqkCQJ4JSlHoAkafkwFCRJjaEgSWoMBUlSYyhIkhpDQZLUrFrqAfw0zjjjjDr33HOXehiSdIdy/fXXH6qq6RP13aFD4dxzz2V2dnaphyFJdyhJvnprfS4fSZIaQ0GS1BgKkqTGUJAkNYOEQpJ7JHl3kmuTvDrJ6iS7k+xPsjOdsWpDjE+SdGJDzRSeAXysqh4JPAB4DnCwqjYAa4DNwLYxa5KkCRkqFL4L3DPJXYDTgEcAV/d9e4GNwKYxa5KkCRkqFN4FPA74EvA5YC1wpO872m+vG7O2QJLtSWaTzM7NzQ00fElamYa6eO0S4LKq+qckbwPuCkz1fVPAIeCeY9YWqKodwA6AmZmZn/obgh7y0rf+tLvQndD1l/7eUg9BWhJDzRTuBdzUt48BbwO29NubgH3AnjFrkqQJGSoUXg88L8lH6c4pvAVYn+QAcJju4L9rzJokaUIGWT6qqhuAR46Ut45sHxuzJkmaEC9ekyQ1hoIkqTEUJEmNoSBJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqDAVJUmMoSJIaQ0GS1BgKkqTGUJAkNYaCJKkZJBSSPDrJNf3j60kuSrI7yf4kO9NZPU5tiPFJkk5skFCoqg9W1flVdT5wAJgCDlbVBmANsBnYNmZNkjQhgy4fJbk7cD/g4cDVfXkvsBHYNGZNkjQhQ59T2AzsAdYBR/raUWDtSdQWSLI9yWyS2bm5uQGHLkkrz9Ch8ERgN3CIbgmJ/uehk6gtUFU7qmqmqmamp6cHHLokrTyDhUJ/kvjRdMtAe4AtfdcmYN9J1CRJEzLkTOGhwGer6iZgF7A+yQHgMN3Bf9yaJGlCVg2146q6DnhS3z4GbB15yrg1SdKEePGaJKkxFCRJjaEgSWoMBUlSYyhIkhpDQZLUGAqSpMZQkCQ1hoIkqTEUJEmNoSBJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSc1goZDkZUk+luTKJPdOsjvJ/iQ701k9Tm2o8UmSbmmQUEjyc8ADqurhwJXAhcDBqtoArAE2A9vGrEmSJmTVQPt9DLAmyYeAbwM/At7R9+0FNgLnAFeMUbtqoDFKkkYMtXw0DcxV1a8C9wXOBI70fUeBtcC6MWsLJNmeZDbJ7Nzc3EDDl6SVaahQOAp8oW9/GXg0MNVvTwGH+sc4tQWqakdVzVTVzPT09CCDl6SVaqhQuB6Y6dv3Ay4BtvTbm4B9wJ4xa5KkCRkkFKrqo8B3knyCbsbwOmB9kgPAYbqD/64xa5KkCRnqRDNV9byR0taR7WNj1iRJE+LFa5KkxlCQJDWGgiSpMRQkSY2hIElqDAVJUmMoSJIaQ0GS1BgKkqTGUJAkNYaCJKkxFCRJjaEgSWoMBUlSYyhIkhpDQZLUGAqSpMZQkCQ1g4RCkocmOZjkmv6xIcnuJPuT7Exn9Ti1IcYnSTqxoWYKa4DLqur8qjofeChwsKo29H2bgW1j1iRJE7JqoP2uAZ6a5MnA14EfAO/o+/YCG4FzgCvGqF010BglSSOGmil8EfizqnoYcBZwAXCk7zsKrAXWjVlbIMn2JLNJZufm5gYaviStTEOFwg3AB+a1bwam+u0p4FD/GKe2QFXtqKqZqpqZnp4eYuyStGINFQovAi5McgpwHvBiYEvftwnYB+wZsyZJmpChQuEfgYuBjwPvAt4ErE9yADhMd/DfNWZNkjQhg5xorqpvAo8eKW8d2T42Zk2SNCFevCZJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqDAVJUnObt7lI8vPAj+nucnqLbmBVVX1piIFJkiZvsXsfXQNcSRcAjwXeP/LzfcCzhxygJGlyFguFz1fVswGS7Kuqi0d+GgiSdCey2DmFOkG7TvRESdId32KhkImMQpK0LJzMTOFk+iRJd0CLhcJDknw4ydXA/ZNcNe/nL/Z1SdKdxGInmtdX1Y231pnE5SVJuhO5zZlCVd2Y5MwkM7fylFurA5DkRUk+kGR1kt1J9ifZmc5YtZ/4nUmSTto4VzSfDbw1yeVJXp7kgiSnJXkC8Npbe1GSc4CL+s1twMGq2gCsATafRE2SNCG3GQpJzqK7mvltwMuAdwIbgBuAvwKecBsvfx1wSd/eBBw//7AX2HgSNUnShCx2TuGVwAOB/wEOAecBdwV+B3g+8HC6q5oXSPJ0YD/w2b60DjjSt48C9z+J2ui+twPbAc4+++xFhi9JOhmLhcIf0x2cLwEeANxQVZcCJPkS8N4kH6yqm0Zet5Vu2emxdAf2m4Gpvm+KLmDuOWZtgaraAewAmJmZ8WOxknQ7WuycwnOA59Et45wDXJXkPUmeCbwdeNYJAoGqenpVnQ9cCFwPvBTY0ndvAvYBe8asSZImZLFQOB04E7gv8AW6K5zvQfdf/Kl0S0Tj2AWsT3IAOEx38B+3JkmakMWWj64E7gX8CV04PJFuOeiX6WYKLwEuvbUXV9UNwK/3m1tHuo+NWZMkTchiM4Xz6b5P4ePAvwOfBL4GfLqqXg08Ksldhh2iJGlSbnOmUFWvBEjyReCrdJ8O2llVl/VPeQXeNE+S7jQWWz4CoKq+3Df/G7hsXv3jQwxKkrQ0/I5mSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqDAVJUmMoSJIaQ0GS1BgKkqTGUJAkNYaCJKkxFCRJzSChkGRVkn9Jcm2Sy5OsTrI7yf4kO9MZqzbE+CRJJzbUTOE3gf1V9UjgLOAFwMGq2gCsATYD28asSZImZKhQeB/w90lWAacDDwau7vv2AhuBTWPWJEkTMkgoVNWNVfU94Frg23Tf7Xyk7z4KrD2J2gJJtieZTTI7Nzc3xPAlacUa6pzCuiR3Ax5Btwx0HjDVd08Bh/rHOLUFqmpHVc1U1cz09PQQw5ekFWuo5aMXA0+rqh8D3wNeAWzp+zYB+4A9Y9YkSRMyVCi8Hnh2ko8C3wHeBKxPcgA4THfw3zVmTZI0IauG2GlVfYPuP/35to5sHxuzJkmaEC9ekyQ1hoIkqTEUJEmNoSBJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqDAVJUmMoSJIaQ0GS1BgKkqTGUJAkNYaCJKkZLBSSvCXJx5K8J8k9k+xOsj/JznRWj1MbanySpFsaJBSSnA+sqqqHA/cGng0crKoNwBpgM7BtzJokaUJWDbTfbwOv69unAH8JPKff3gtsBM4BrhijdtVAY5SWva/99QOXeghaZs7+888Muv9BZgpV9V9VdV2SpwA3A58CjvTdR4G1wLoxawsk2Z5kNsns3NzcEMOXpBVryHMKTwJeCDwR+BYw1XdNAYf6xzi1BapqR1XNVNXM9PT0UMOXpBVpqHMK9wFeCmytqv8F9gBb+u5NwL6TqEmSJmSomcJFwFnA+5NcA5wKrE9yADhMd/DfNWZNkjQhg5xorqpXAa8aKb9xZPsYsHWMmiRpQrx4TZLUGAqSpMZQkCQ1hoIkqTEUJEmNoSBJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqDAVJUmMoSJIaQ0GS1BgKkqRmsFBIcmqS9/bt1Ul2J9mfZGc6Y9WGGp8k6ZYGCYUkpwHXA5v70jbgYFVtANb09XFrkqQJGSQUqur7VfUg4GBf2gRc3bf3AhtPoiZJmpBJnVNYBxzp20eBtSdRWyDJ9iSzSWbn5uYGHbQkrTSTCoVDwFTfnuq3x60tUFU7qmqmqmamp6cHHbQkrTSTCoU9wJa+vQnYdxI1SdKETCoUdgHrkxwADtMd/MetSZImZNWQO6+q+/U/jwFbR7rHrUmSJsSL1yRJjaEgSWoMBUlSYyhIkhpDQZLUGAqSpMZQkCQ1hoIkqTEUJEmNoSBJagwFSVJjKEiSGkNBktQYCpKkxlCQJDWGgiSpMRQkSY2hIElqll0oJFmdZHeS/Ul2JslSj0mSVoplFwrANuBgVW0A1gCbl3g8krRiLMdQ2ARc3bf3AhuXcCyStKKsWuoBnMA64EjfPgrcf35nku3A9n7zxiRfmODY7uzOAA4t9SCWg7zmoqUeghbyb/O4v7hdVtTPubWO5RgKh4Cpvj3FyB9CVe0Adkx6UCtBktmqmlnqcUij/NucnOW4fLQH2NK3NwH7lnAskrSiLMdQ2AWsT3IAOEwXEpKkCVh2y0dVdQzYutTjWKFcltNy5d/mhKSqlnoMkqRlYjkuH0mSloihIElqDAV5axEte0lOTfLepR7HSmAoCLy1iJaxJKcB1+Pf5UQYCgJvLaJlrKq+X1UPAg4u9VhWAkNBcMtbi6xdwrFIWkKGgmCRW4tIWjkMBYG3FpHUMxQE3lpEUs8rmiVJjTMFSVJjKEiSGkNBktQYCtJPIMldxu0bvW1IktW38rpTvcWIltqy+z4FaTlIMgvcOFK+V1U9pG9fnuT499wGeCRwTb/9HeCp/X6mgN3Ao+bt57okD66qH43s/8nArwN/mOTDwOaquul2eUPSmAwF6cRGD9gAPzzeqKqLjreT/A3w+ap67vwnJ7kYeB6wJsn7gTcCfwD8LPCvSVYBT62q/+tf8nTg7/r2saq6Kck64GFVdeXt9L6k22QoSCd2CvCskdrb528kuQdwKXAB8MIk7wEuO34Ar6o3J/kVuutADtDNPD4CXFpVz0xyalX9sN/XecCTq+qCkd/5t8DXAENBE+F1CtIJJHkucNZI+VtV9Yb+v/eLge3APwB3A24APkf3n/4pdDOEOeBy4JPAp4GH0i0RTQFf6Z93OfBOuhsR3qeqzu1//weAtwKPr6rfHehtSrfgTEGaJ8n9gDcAx9fy79b/PNb3PxX4IDAN/FpVfTPJSwCq6j+AxyW5ECi67xr/JeAXgMfTLR19G/hGVbXvBkhyOvAa4AXzhrIauJBuFiJNjDMF6QSSPJ7uHlC/Bayqqn/uzwHcXFU398/ZC9yFLgCOW1dVD+z7VwE3Ay8DPgN8gC4YngKcDpxTVdPzfucHgd+nWzJ6LHBmVf1gwLcp3YIzBWlE/5HS1wIP6EsvTPIUugB4OfDxvv5DYMv8TxEl+di8XT0NuAg4EziPbonpGPAMYCddAIz6GbpZw7rjgZDk9Kr67u3w1qRFGQrSLf023XmAXcD36db930D30dOpJFNVdYRuhrAnyfyZwmnz2u+sqrcluQT4FPB5uo+c/ifwR1W1f+T3pqo+ApCEJGur6jDw5iSvqaprb/+3Ki1kKEjzJLk78HzgN+jOKzyF7rbi/wbcnS4Y/hT4EHAq8JiRmcInjrer6ljfPJVuCerHwOuSfIjuQP+ekRnAaUlW9fu7Anh3fzHbV4Drhni/0ijPKUgj5h2YF3veaVX1/Z/wd6z2wjQtR4aCJKnx3keSpMZQkCQ1hoIkqTEUJEmNoSBJav4fODXVIa5UsH4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt #导入pyplot模块\n",
    "import seaborn as sns #导入Seaborn\n",
    "fig = sns.countplot('是否转化', data=df_fission) #创建柱状计数图\n",
    "# fig.set_xticklabels(fig.get_xticklabels(),rotation=25) #X轴标签倾斜\n",
    "fig.set_ylabel(\"数目\") #Y轴标题\n",
    "plt.show() #显示图像"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建特征和标签数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# df_fission = df_fission.query(\"裂变类型 == '助力砍价'\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_fission.drop(['用户码','是否转化'], axis = 1) # 构建特征集\n",
    "y = df_fission.是否转化.values # 构建标签集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>性别</th>\n",
       "      <th>年龄</th>\n",
       "      <th>入会月数</th>\n",
       "      <th>会员类型</th>\n",
       "      <th>活跃用户</th>\n",
       "      <th>会员积分</th>\n",
       "      <th>总消费</th>\n",
       "      <th>年收入</th>\n",
       "      <th>城市_杭州</th>\n",
       "      <th>城市_苏州</th>\n",
       "      <th>裂变方案_疯狂打折</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>516</td>\n",
       "      <td>6360.66</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>69</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>682</td>\n",
       "      <td>28605.00</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>803</td>\n",
       "      <td>10378.09</td>\n",
       "      <td>236311.0932</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>778</td>\n",
       "      <td>25564.01</td>\n",
       "      <td>129909.8079</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>547</td>\n",
       "      <td>3235.61</td>\n",
       "      <td>136976.1948</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>1</td>\n",
       "      <td>49</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>746</td>\n",
       "      <td>1930.44</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>569</td>\n",
       "      <td>29593.82</td>\n",
       "      <td>82517.8587</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>0</td>\n",
       "      <td>55</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>850</td>\n",
       "      <td>32841.84</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>1</td>\n",
       "      <td>35</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>529</td>\n",
       "      <td>21564.34</td>\n",
       "      <td>117800.7531</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>0</td>\n",
       "      <td>48</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>657</td>\n",
       "      <td>19371.82</td>\n",
       "      <td>176622.9201</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      性别  年龄  入会月数  会员类型  活跃用户  会员积分       总消费          年收入  城市_杭州  城市_苏州  \\\n",
       "0      0  40     9     2     1   516   6360.66       0.0000      1      0   \n",
       "1      1  69     6     2     1   682  28605.00       0.0000      1      0   \n",
       "2      0  32     9     1     1   803  10378.09  236311.0932      1      0   \n",
       "3      0  37     0     2     1   778  25564.01  129909.8079      0      1   \n",
       "4      1  55     4     3     0   547   3235.61  136976.1948      0      1   \n",
       "...   ..  ..   ...   ...   ...   ...       ...          ...    ...    ...   \n",
       "9995   1  49     7     2     1   746   1930.44       0.0000      0      0   \n",
       "9996   1  36     1     1     0   569  29593.82   82517.8587      1      0   \n",
       "9997   0  55     7     1     0   850  32841.84       0.0000      0      0   \n",
       "9998   1  35     5     1     1   529  21564.34  117800.7531      0      0   \n",
       "9999   0  48     5     1     0   657  19371.82  176622.9201      0      1   \n",
       "\n",
       "      裂变方案_疯狂打折  \n",
       "0             1  \n",
       "1             1  \n",
       "2             0  \n",
       "3             0  \n",
       "4             0  \n",
       "...         ...  \n",
       "9995          1  \n",
       "9996          0  \n",
       "9997          1  \n",
       "9998          0  \n",
       "9999          1  \n",
       "\n",
       "[10000 rows x 11 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler #导入归一化缩放器\n",
    "scaler = MinMaxScaler() #创建归一化缩放器\n",
    "X_train = scaler.fit_transform(X_train) #拟合并转换训练集数据\n",
    "X_test = scaler.transform(X_test) #转换测试集数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 0.37837838, 0.2       , ..., 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [1.        , 0.27027027, 0.5       , ..., 1.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.        , 0.28378378, 0.7       , ..., 0.        , 0.        ,\n",
       "        1.        ],\n",
       "       ...,\n",
       "       [0.        , 0.68918919, 0.9       , ..., 0.        , 0.        ,\n",
       "        1.        ],\n",
       "       [0.        , 0.13513514, 0.9       , ..., 1.        , 0.        ,\n",
       "        0.        ],\n",
       "       [1.        , 0.10810811, 0.7       , ..., 0.        , 1.        ,\n",
       "        0.        ]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix # 导入混淆矩阵\n",
    "import seaborn as sns #导入seaborn画图工具箱\n",
    "def show_matrix(y_test, y_pred): # 定义一个函数显示混淆矩阵\n",
    "    cm = confusion_matrix(y_test,y_pred) # 调用混淆矩阵\n",
    "    plt.title(\"混淆矩阵\") # 标题\n",
    "    sns.heatmap(cm,annot=True,cmap=\"Blues\",fmt=\"d\",cbar=False) # 热力图设定\n",
    "    plt.show() # 显示混淆矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 决策树测试准确率: 84.20%\n",
      " 决策树测试F1 分数: 54.20%\n",
      " 决策树Bagging 测试准确率: 85.15%\n",
      " 决策树Bagging 测试F1 分数: 56.52%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAOvklEQVR4nO3de7Sd85nA8e+TnNxDIsQY4i5h6CqGSLQqLtWyOloGRUPcM6Yoy1KXKVNDXFqKzrh0MlEJieu0bmXQEiFVkpO6E4lOFyKIkEhEnDj85o/zRoMkZThn7+ec72etvc5+877n3c8+a6/v+a1370OUUpAk1b9OtR5AkvTpGGxJSsJgS1ISBluSkjDYkpSEwVbdioiGz3Bs/4jYorrfLSJOjIi1qu1hETHm4+eLiFMjou/nmK9vRFwYEZ3/v+eQPguDrXp2WESMi4iuEfHDiJgREZOXub0YEftWx/4d8NuIGE7L63oBcEtE3A9cBIwqpTQvPXFEDAQOqY5b+m8PRETjMrcnImKTat+0anteROxTfcsw4EullPdb+wchAXzqFYxUA78EtgbOAV4Czi2ljF26MyLOBJYAlFIeiIhdgQFA/+o2H7ga2AkYGxE/K6XcXn37AbS8/q+NiE7AfwGDgAFLw16tyJfGuAnYB7gOeC0iHgfWAt6IiMlAAKsC25RSlnzRPwgJDLbqW7dSyvcjYhVaVsP/EhFHLrN/PaAxIgLYsZQyKSKmA2cDjwJ/BlYrpYyMiI2AVQAiogdwKLAv8N/AnsAc4N1lH3zZFTnwAS2r+GdLKZMjYjfgVuDw6jEe+oKfu/QJBlt1qboufE9EXFtKuTwiurL8FXYnWlbTF0bEbOBi4GvAdsBWwIsR8Z3qW3pGxCjgCeDHpZRHI+K9Usoz1fkAHq6+rgLcWko5eZmxBgELq/vn0PKLYWtgS8Bgq9UZbNWlUsr71Sr2nIjoD/QFDqxW2H2B7sCrwKRSypyIGArsV0q5HxgWEacCPYDTljntM6WUN6s3I9eIiGOBvhHxA+Cx6pihpZTmiDgUGPixsW4FromIYcARwOBqls4RsSPw51LK977In4O0LIOtulVKWRwRV5dSXo+IwcD+wCxaLmUMAu4ENqmOfT8ingWIiG8Co6rbVtXp9gPOA+6i5br0PFqiv5DlXA5ZidtpWVVvBrwI7AVsVUo59fM8V+nTMNiqWxHRDbiruv78Pi1v+L0HXAJ8G9gCuKA69u+BMRFxJXAS8CSwB7C4Ot0mQDNAKeV1YFxEfIuWaN9b/VKAv1wSAbjmYyNtDuwK3FZKmVE97hf7pKWVMNiqZzvREtN3qjB+B1gEfBO4u5RyWkSsU33KY2/gRuB+4A7gUuCEUsrzABFx6XLOfzBwPXBH9QkTqC6JrGCeLsA9wOKImEbLL4M1gF4RsQPQCxhZSpn6+Z62tHwGW/XsMODO6g3HzWj56N0HtHxUj+rTIVcAPwdGAF8vpcys9nUCJkTE0hX2QODmpSeOiJ2BgaWUAyLiAlreQOy67INXjxullCZaYv14KeXWavcvq2MOwEsiaiMGW/XsNeB31f3JwCWllMciYkPgKGAiLde05wIvLI11ZVVgeCnl+er697nA0pj3AcbTsmIHOBPoTMvH9u5f5jJHZ+BC4Fe0BHt5ulU3qdWF/wMDdUQRsXYpZXat55A+C4MtSUn43xKRpCQMtiQlYbAlKYlW+5RIj62P9eK46tK8qcv7SLZUH7o3sMK/xnKFLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSqKh1gOoRUNDJ66/8Cj2PeE/2Wbz9bjhoqN4YfabABz9bxOY+cKcjxwD0K1rA9decCQD1urLUzNmc8QZV9fyKaiDmDrlES7990sAeOWV2Rx73Ak8MGkir8+Zw/obbshZo86r7YDt2EpX2BHx9Yg4JyKuiIizI2KXthqsI+nerQsPTTiFXYduBkDfVXsy+qbJ7Hr4xex6+MXMfGHOJ44BOPBbg3l5znyG7H8+fVft+ZF9UmsZvN0Qxo2/jnHjr2PQoE2ZPftlBm26GeMmXM/c119n+rPP1nrEdmuFwY6IccDxwFygEZgHnBgRY9tmtI7j3ab32G7/83j5tfkArLZqT/badSsevOYkrrvwyOUeA7DT4E257+HpAEyaOoNhgwe19ejqwBYvXsyLL77AiEMO4+BDDqO5uZmFCxfSu3fvWo/Wbq3sksiXSylbf+zfLoqIx1pxHgF/eul1zrr8N9w1+Wkmjj2Rr20zkAenzfzEcf369OKttxcDsGDRYgauv2Zbj6oO7OE//J4hQ7anZ69eAAw/YD/69+/PgHXXrfFk7dfKLonMiojLI2LviNil+noF8NKKviEiRkZEY0Q0Ns99+ouftoN4Yfab3PfIcx/eX7Pf8lcsb8x/mz69ewDQp3cP3pi/qM1mlCbdP5Edd9qZ+fPnsWTJEq6ecD0LFixgyiMP13q0dmtlwd4XmFZ9PRnYB5hSfV2uUsroUsq2pZRtG9bY4gsdtCP5wUG7sN/u2xARbL7x3/L0n15Z7nETpzz34XXrYYMHMalxRluOqQ6slELjlClsN2QoV4+9invu/h86d+5M9+7daWpqqvV47dYKg11KaSqlXFlKGV5K2b2UclAp5apSypK2HLAj+sUNkxjx7aE8eM1J3Dbxcab/76vLPe76OxtZe82+TLnhNOa99Q4Tq1W51NqeevJJNtp4Y7p168b+Bw7nll//ioO/tz99+vblK1/dodbjtVtRSmmVE/fY+tjWObH0Oc2bemmtR5BWqHsDsaJ9/uGMJCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpiSiltMqJZ81rap0TS59TQyfXKapfa/XpEiva5ytXkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2HXm/LN+xLFHDOf0k47j/ebmT2wvddO1V/PDY4+q4aTqiJqb3+PUE48BYOGCtzj+6EM55siDGHflL6r9zfzrqSdyzJEHcf7Zp9dy1HbJYNeRJx/7Ix80v8+lV07gnUWLmDBuzEe2G6f8AYDXXpnNPXfeVuNp1dE0vfsuR434LtOq1+Hv7r6TDTbahMvGjOepxx/llZdnMXnSfWwycFMuGzOeN+bOZeaM6TWeun0x2HVktX6r84/7Dwfgg/IBvXr3/sj2Upde/BOO/P7xNZlRHVe37t256tqb6b/m3wBQSuGdRYsopVBKYeaM6Wy3/Q58d/gImpubeXvhQnr16lXjqduXhloPoL8YsN76AEy+/146RSf22ucAOjc0fLi97Xbbc+/dd7DxwEGsv8FGNZ5WHd1ue/wDUx95iDNOOYEuXbvS1NREz549ATj6sANZffX+rL3OujWesn1Z6Qo7Im6LiHciYsYyt5kRMWMFx4+MiMaIaJwwdkzrTNzOPfTARH5947WMuvA/6NzQ8Inth3//AI9OncKoM05mxnPPcMtN19V6ZHVgp5x+FqN++nO6dOnKav368db8+SxZsoTLxoxn4cIF/LFxSq1HbFf+2gp7H6CxlLLlpzlZKWU0MBpg1rym8jln63DefGMuN0wYy/mXXEGPHj0/sQ3wo7N+AsCrs1/mZ+eeyV77HVjLkdWBPfHoNO6641bOOPunPD9zOpt/aUvGjx3NBhtuzDf22JNu3buzpOndWo/Zrqw02KWU9yJiSFsN09HdfcdtvPnGXE49/p8BGLrDsI9s777nXuyx5961HFH60JCv7MBtN9/IcSNHMOLwf6Jnz57sve+BnPPj07j5putZZ8AABg/9aq3HbFeilNZZCLvCVr1q6OR77apfa/XpEiva5ytXkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCSilFLrGfQpRMTIUsroWs8hfZyvzbbjCjuPkbUeQFoBX5ttxGBLUhIGW5KSMNh5eI1Q9crXZhvxTUdJSsIVtiQlYbDrXER0j4jfRMTjEXFNREStZ5KWFRFdIuL2Ws/RERjs+ncQMKuUsiWwGrBbjeeRPhQRPYBp+LpsEwa7/u0C/La6fx+wcw1nkT6ilLK4lPJlYFatZ+kIDHb9Wx14q7q/AOhXw1kk1ZDBrn9zgT7V/T7VtqQOyGDXv3uBb1T3dwEm1nAWSTVksOvfBGCdiHgCeJOWgEvqgPzDGUlKwhW2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlMT/AfAxpiyw8HhAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 对多棵决策树进行聚合(Bagging)\n",
    "from sklearn.ensemble import BaggingClassifier # 导入Bagging 分类器\n",
    "from sklearn.tree import DecisionTreeClassifier # 导入决策树分类器\n",
    "from sklearn.metrics import (f1_score, confusion_matrix) # 导入评估指标\n",
    "dt = BaggingClassifier(DecisionTreeClassifier()) # 只使用一棵决策树\n",
    "dt.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = dt.predict(X_test) # 进行预测\n",
    "print(\" 决策树测试准确率: {:.2f}%\".format(dt.score(X_test, y_test)*100))\n",
    "print(\" 决策树测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "bdt = BaggingClassifier(DecisionTreeClassifier()) # 树的Bagging\n",
    "bdt.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = bdt.predict(X_test) # 进行预测\n",
    "print(\" 决策树Bagging 测试准确率: {:.2f}%\".format(bdt.score(X_test, y_test)*100))\n",
    "print(\" 决策树Bagging 测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_bdt = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 随机森林测试准确率: 86.90%\n",
      " 随机森林测试F1 分数: 60.90%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPQElEQVR4nO3de7SVdZnA8e8DKLfkcpSykbREyLSlEmo0k2igZau8FDpSSJIalcvwkhnmlCaipRg2Mml4AxXMrDFBHS+jIiI1CIk2KZFNiogXLiqIwhH4zR97Y6QcspSz93PO97PWXmcf9nve/Zyz3vXdP979HohSCpKk+tem1gNIkt4agy1JSRhsSUrCYEtSEgZbkpIw2KpbEdHu79i2R0TsXr3fPiJOjYjtq5/vHxFXvHF/ETEqIrq9jfm6RcTYiGj7j+5D+nsYbNWzL0fEpIjYOiK+FRELImLmRreFEXFEddsPAXdFxFAqx/UK4FcRMR34EXBuKWXthh1HRG/gmOp2G/5sRkTM2ej2SETsUn1sbvXzFyJicPVL9gc+XEpZt6V/EBLAW17BSDVwFdAXGAM8BZxXSpm44cGIOBtoBCilzIiIQUBPoEf19iJwDXAAMDEiLiqlTKt++RAqx/+UiGgDXA70AXpuCHt1Rb4hxmuAwcD1wHMR8TCwPbAsImYCAXQB+pVSGt/pH4QEBlv1rX0p5YSI2IbKavg7EXH8Ro/vCMyJiAAGlFLui4j5wGjgIeDPQPdSyoiI2BnYBiAiOgLDgSOAXwCHAM8Dqzd+8o1X5MB6Kqv4x0opMyPiIOBm4Njqc8x6h7936U0MtupS9bzwnRExpZTyk4jYmk2vsNtQWU2PjYjFwDhgP2BfYC9gYUQcVv2SThFxLvAIcFYp5aGIeK2U8mh1fwC/qX7cBri5lHL6RmP1AVZW74+h8sLQF9gTMNja4gy26lIpZV11FTsmInoA3YAvVFfY3YAOwLPAfaWU5yOiP3BkKWU6sH9EjAI6AmdstNtHSynLq29GbhcRJwLdImIkMK+6Tf9SytqIGA70fsNYNwPXRsT+wHHAPtVZ2kbEAODPpZQvvpM/B2ljBlt1q5TyakRcU0pZEhH7AEcBi6icyugD3AbsUt12XUQ8BhARnwLOrd72qu7uSOB84HYq56VfoBL9lWzidMhmTKOyqt4VWAgcDuxVShn1dr5X6a0w2KpbEdEeuL16/nkdlTf8XgMuBg4FdgcurG77EeCKiLgSOA34HfBp4NXq7nYB1gKUUpYAkyLiM1SifXf1RQH+ckoE4No3jLQbMAiYWkpZUH3ed/abljbDYKueHUAlpq9Uw3gYsAr4FHBHKeWMiNihepXH54CfA9OBW4HxwMmllMcBImL8JvY/DPgZcGv1ChOonhJpYp6tgDuBVyNiLpUXg+2AzhHxcaAzMKKU8uDb+7alTTPYqmdfBm6rvuG4K5VL79ZTuVSP6tUhlwI/Br4EHFhK+WP1sTbA5IjYsMLuDdy0YccR8QmgdyllSERcSOUNxK03fvLq80YpZQ2VWD9cSrm5+vBV1W2G4CkRNRODrXr2HPDf1fszgYtLKfMi4gPAV4B7qZzTXgo8uSHWVV2AoaWUx6vnv88DNsS8K3AdlRU7wNlAWyqX7U3f6DRHW2As8Esqwd6U9tWbtMWF/4GBWqOI+KdSyuJazyH9PQy2JCXhvyUiSUkYbElKwmBLUhJb7CqRjn1P9OS46tILD27qkmypPnRoR5O/jeUKW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBrhPt2rXhFxd/FYB+u+3I47eP5u6rTuHuq06h907vpm3bNky+4FjuufoULjtr6F997cijB3LrZSfWYmy1YtdMvJoRxw1nxUsvcdzwYRwzdAg/vfQ/aj1Wi7bZYEfEgRExJiIujYjRETGwuQZrTTq034pZk7/NoP67AtCtSycm3DiTQceOY9Cx4/jjk89z6Cf24JEFTzPwy+PYvkcX9uizAwA7vrc7Qw/Zt5bjqxVavPhppt18EwC33XoLvXrtwqTJP2PeQ79l0aKnajxdy9VksCNiEnASsBSYA7wAnBoRE5tntNZj9ZrX2Peo83n6uRcB6N6lE4cP2ov7rz2N68ceD8CdDzzKv193D23btqHbNp1YsWo1AGO/dQTfu2RqrUZXK3XB+WMYeco3ASilsOqVVZRSKAX+MH9+jadruTa3wt6jlHJIKWVcKeXKUsqPSimfBfZqptlarT89tYRzfnIL+w0by/bbdWG/fr1Z9Wojr65+jXuvPpXnl63giaeXcdTBe/PIgqd57E/P1npktSK33TKNPh/clZ179QLgM4ccysoVKzj15G+w9dZbsWb16hpP2HK128xjiyLiJ8BdwEtAV+CTQJN/34mIEcAIgHY9D6Dddru/g6O2Hk8uXs7vH3/m9fvvbngXDV078/Irazhg+EXcPmEkA/buzacHfJj3bd+dgz72IXq//z187agBXHbDjBpPr5Zuxn3TeeaZxcx6YCZPPPFnfjblOs4efR4NDQ2cdspIGrbdttYjtlibW2EfAcytfjwdGAzMrn7cpFLKhFLK3qWUvY31P27k0QM58uB+RAS79Xovv//TM5w0bCCfP6gv69cXXlndSMcOWzH8OxMZdOw4ho26moceW2is1Sx+cOFFTLruen449kfsttvu9OnzQc79/lk0Njbyh/nz2WOPPWs9YovVZLBLKWuqp0KGllIOLqUcXUq5upTS2JwDtkaX3XAfXzq0P/dfexpT732Y+f/3LD+9YQbHHNaf6ZO+yfIXV3HXrMdqPaYEwL/sN4DGxjUMH/ZFRnztBDp17lzrkVqsKKVskR137Hviltmx9Da98OD4Wo8gNalDO6Kpx7wOW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlESUUrbIjhcuX7Nldiy9TW2i1hNITevZvX2TR6grbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKol2tB9Bfu+CcM3lq4RN0697A2eePo1D4/qhTGD12PADr1q5lzPe+zbKlS3jfju/ntH87p8YTq7X4wTlnsujJyrF55jk/ZPR3T2fJc8+y8y59GHXWGCICgBunXMPsWfdz4fjLazxxy+MKu47878O/Zd26dVxyxWReWbWKOf8zixOGD2Hug795fZsHZtxDr94f5McTrmH5siU8vmB+DSdWa/G7eb9l/dp1jL+ycmz+17Sb6PHu93D5db9g5coVzJ39awCee2Yxd942tcbTtlwGu450a9iWz/3rUABKWQ8RTLjul/To8Z7Xt9mn/8cZ/IVhrFu7lpdfXknnzu+q1bhqRbo3bMvnj6ocm+vLeq658lL67dMfgL799mXe3AcBGD/uhxx/wkk1m7Ol85RIHen5vp0AmDn9biLasPe+H3vTNh07dQLgG8d9kYbtevDeHXo264xqnXru+Jdjs020YZc+u9L5XdsA0KlzZ55a+AR333ErvXr3Yaf371zLUVu0za6wI2JqRLwSEQs2uv0xIhY0sf2IiJgTEXOmTLpiy0zcws26/15+deMURl94CW3bvfn1dMVLL9LY2MjFE65l5YoVzJs7uwZTqjWaNeNe/vPnUzh37CU0NGzHqpdXArDq5Zfp2rUbv3lgBg89OJtzv3s6C/7wKL+68foaT9zy/K0V9mBgTillz7eys1LKBGACwMLla8rbnK3VWb5sKTdOnsh54y6lY8dOm9zmximT2OkDvTjw4M/SoUMH1qxZ08xTqjVavmwpN0yeyA8urhybfff5KHNm/5oBAw9i3tzZDB4yjH77Vk6RPLv4aS4672wOP/ILNZ665dnsCruU8hrw0WaapdW787apLF+6lDNO/jonf/UYbp9205u2OWzwEO645SZGfuVounTtxt4f/ecaTKrW5o5bp7J82VJGnfR1ThpxDGvXrmXpkuc5fuhgtunSlY/sYyaaQ5SyZRbCrrBVr9pErSeQmtaze/smj1CvEpGkJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpiSil1HoGvQURMaKUMqHWc0hv5LHZfFxh5zGi1gNITfDYbCYGW5KSMNiSlITBzsNzhKpXHpvNxDcdJSkJV9iSlITBrnMR0SEibomIhyPi2oiIWs8kbSwitoqIabWeozUw2PXvaGBRKWVPoDtwUI3nkV4XER2BuXhcNguDXf8GAndV798DfKKGs0h/pZTyaillD2BRrWdpDQx2/dsWeKl6fwXQUMNZJNWQwa5/S4Gu1ftdq59LaoUMdv27G/hk9f5A4N4aziKphgx2/ZsM7BARjwDLqQRcUivkL85IUhKusCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQk/h9B0rghRKAauwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV # 导入网格搜索工具\n",
    "from sklearn.ensemble import RandomForestClassifier # 导入随机森林模型\n",
    "rf = RandomForestClassifier() # 随机森林模型\n",
    "rf.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = rf.predict(X_test) # 进行预测\n",
    "print(\" 随机森林测试准确率: {:.2f}%\".format(rf.score(X_test, y_test)*100))\n",
    "print(\" 随机森林测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_rf = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 极端随机森林测试准确率: 85.05%\n",
      " 极端随机森林测试F1 分数: 54.90%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPP0lEQVR4nO3de5RWdbnA8e8DAwwgCKh5YRJvkEEppJanZZnazeXJNC9576LSqVMWZaXWOZX3oxy0k7c0RO5qukzN8n43SyEvR61jmIgeixxAQUAuw+/88W5qMoZjCfO+D/P9rPWu2ePes99nxnd957f2u4EopSBJanzd6j2AJOmNMdiSlITBlqQkDLYkJWGwJSkJg62GFRFNf8exm0XEiGq7V0R8NSK2qD7fMyJ+9PrzRcRJETHgTcw3ICLGRkT3f/Qc0t/DYKuRfSYiJkZEz4j4ekQ8HRH3t3vMiYiDq2PfDtwWEUdSe10vBH4SEXcD44DTSykrV584IoYCn6qOW/3f7o2IGe0ej0fEDtW+mdXnCyLioOpL9gTeUUppW98/CAngDa9gpDq4HBgFnAE8D5xZSrli9c6I+C6wHKCUcm9E7AO0AJtVj5eBScAHgCsi4j9LKTdWX34Ytdf/tIjoBlwGDANaVoe9WpGvjvEy4CBgOjA3Ih4DtgDmRcT9QAD9gV1KKcvX9Q9CAoOtxtarlPKFiOhHbTV8SkQc127/1sCMiAjg/aWUeyLit8BpwCPAs8DAUsroiNgO6AcQEb2BTwMHA9cAHwP+BLzW/snbr8iBVdRW8b8ppdwfER8Crgc+Wz3HL9bx9y79DYOthlRdF741IqaVUi6KiJ6seYXdjdpqemxEvAicB7wPeDcwEpgTER+vvqRPRJwOPA58p5TySESsKKU8VZ0P4JfVx37A9aWUb7QbaxiwqNo+g9ovhlHAzoDB1npnsNWQSilt1Sr2jIjYDBgAHF6tsAcAzcAfgXtKKX+KiN2BQ0opdwN7RsRJQG/g5HanfaqUMr96M3LTiPgiMCAiTgAerY7ZvZSyMiI+DQx93VjXA5MjYk/gWGC3apbuEfF+4NlSyhHr8ucgtWew1bBKKUsjYlIp5aWI2A34JPACtUsZw4CfATtUx7ZFxG8AIuIjwOnVY2R1ukOAs4CbqV2XXkAt+otYw+WQtbiR2qp6R2AOcAAwspRy0pv5XqU3wmCrYUVEL+Dm6vpzG7U3/FYA5wP7AyOAc6tj3wX8KCLGAycC/w3sCyytTrcDsBKglPISMDEi9qMW7TuqXwrwl0siAJNfN9JwYB/ghlLK09XzrttvWloLg61G9gFqMV1ShfHjwGLgI8AtpZSTI2JwdZfHgcDVwN3ATcAFwFdKKbMAIuKCNZz/aOBK4KbqDhOoLol0ME8P4FZgaUTMpPbLYFOgb0TsAfQFRpdSHn5z37a0ZgZbjewzwM+qNxx3pHbr3Spqt+pR3R1yMfB94Bjgg6WU31X7ugFTI2L1CnsocN3qE0fEXsDQUsphEXEutTcQe7Z/8up5o5SyjFqsHyulXF/tvrw65jC8JKJOYrDVyOYCt1fb9wPnl1IejYhtgeOBu6hd024Fnlsd60p/4MhSyqzq+veZwOqYbwxMobZiB/gu0J3abXt3t7vM0R0YC1xLLdhr0qt6SOtd+A8YqCuKiK1KKS/Wew7p72GwJSkJ/y4RSUrCYEtSEgZbkpJYb3eJ9B71RS+OqyEteHhNt2RLjaG5iQ7/NJYrbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGu0E0NXXjmvM/B8Auw7dm1s2nccflY7jj8jEMHfKWvzkGYEC/3txy2Ze5c8IYTjr+o3WZW13ThPGXcdThh/KFzx3HiuXLAZh0xQRGH/vp+g62gWta286I+CCwFzAIaAXuKqXc2RmDdSXNvXpw76QT/xzmAf37cOmP7+ec8bd0eAzAJ/fdjaee+QNjzr6a6y/4AkO22oTnXpzX6fOra3nh+ed5ZtYspky/mqlTJjF37ly6de/Gjddfx8BBg+o93gatwxV2REwEvkwt1DOABcBXI+KKzhmt63ht2Qre/cmz+N+5LwMwsH8fDthnJPdNPpHpY49b4zEAEdCvb69qO9j5bYM7e3R1Qb/65YMsXPgKnznmSB6ZOYPBLS2cc9YZnDDma/UebYO3tksiO5VSPlZKOa+UMr6UMq6U8s/AyE6arct65vmXOPWin/K+o8eyxab9ed8uQ9d43PSbHmbjfn24cuxxLFuxkt69enbypOqKFiyYz8BBg5gwaSpz587lhxdfyLC37ch2229f79E2eGu7JPJCRFwE3Aa8AmwMfBh4vqMviIjRwGiAppYP0LTpiHU4atfx3IvzeXLWH/68/ZZBG3V47Oe/N5XWBa8y9ZzP8qcFizprRHVhfftuxDbbbAtAS0sLl1x0ATuPHMUvHrif2bOfZfrUKRx+5FF1nnLDtLYV9sHAzOrjN4CDgIeqj2tUSrm0lLJrKWVXY/2PO+GovTnko7sQEQzffkuefOYPazxuj1124AffOoyePZrY6W0tPPT4s508qbqi4SNG8NSTTwAwZ84cJk+7molTpvMfY8cxfPgIY70edRjsUsqy6lLIkaWUj5ZSjiqlTCilLO/MAbuiS666h2P23537Jp/IDXc9xm9//8c1HnfLA0/Sq2cTd1z+Fc667GYWL/V/jda/nUeOYuMBAzji0IPYZttteedOO9V7pC4jSinr5cS9R31x/ZxYepMWPHxBvUeQOtTcRHS0z/uwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSUQpZb2cePa819bPiaU3qblH93qPIHVoi/49oqN9rrAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKYmmeg+gv3buad/mhTmzGTBwEP9+5jjO+s5JzGt9iZath/C1b53K/zz1BKeePIbNt9wKgDEnf4+3DtmmvkOry1i5cgXf/vpXOPu8C1m6dAmnffubvPLyAt6x8yg+f8LXADjzu6fw/HO11/Bp55xPU5OZWVdcYTeQJx77NW1tbXz/siksWbyYO2/7OdvtMIzzfjiR+fNaeebp3/LqooXsd+ChjLtkIuMumWis1WmWvfYaxx99KDMfehCA235+E8PfsRMXjp/C7N/PYvazz/D4o7+mbWUbF0+YxpLFrzLjV7+o89QbFoPdQAYO2oQDDj0CgFWrVrHRRv34xOFH07ZyJYsXLaJP341YtGghD9x9O1869ghOPeWrlFLqPLW6il7NzUyYfh2bvWVzADbq14+lS5fQ1tbGsmXL6NGjB4MGbcLBhx8FwCpfm+ucwW4gg986hB2Hv5MH7rmDbt268e5/2oPm5t6M+ZdPMWDQJmw5uIXBLVtzzPH/yg/GT2N+ayuPPzKj3mOri3r/Xvvw0IMPcMSB+zJkm+0Y3LI1LVsP4e0j3sm9d91Otwh2fc976z3mBmWtwY6IGyJiSUQ83e7xu4h4uoPjR0fEjIiYMW3i+PUz8Qbuwfvu5ic/nsb3zvkvFi9+leXLl3PeDyfx6qKFPDrzITbfcitG7bY7AJtvuRUvL5hf34HVZU2Z8CM+ftChXHXDrSxc+ApPPPYIAA/ccxfXXjWVs8Zd6PXrdez/W2EfBPyulDKs3WNoKWXYmg4upVxaStm1lLLrEZ86dt1Pu4GbP6+VH0+7gtPO/QF9+vblmumTuO/OW+nevTu9mptZvmwZ106fzD2338yqVat47vez2Ga7Heo9trqopUsW07NnLwB69ujJ0qVLmNfayvQpEzj7vAvp07dvnSfc8Kz1118pZUVEvKezhunqbvvZDcxvbeWUMZ8HYNfd9+CWn/6EG6+9ii0Ht7DLe97L9sN25OzvnMQN11zJe/fcmyHbbl/nqdVVHXDI4Zz+b9/kumuuZPPNt+Bdu+3OVVOuYH7rS3z9S58DYN/9D2S//T9R50k3HLG+3rSaPe8133FQQ2ru0b3eI0gd2qJ/j+hon286SlISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpRElFLqPYPegIgYXUq5tN5zSK/na7PzuMLOY3S9B5A64GuzkxhsSUrCYEtSEgY7D68RqlH52uwkvukoSUm4wpakJAx2g4uI5oj4aUQ8FhGTIyLqPZPUXkT0iIgb6z1HV2CwG99RwAullJ2BgcCH6jyP9GcR0RuYia/LTmGwG9/ewG3V9p3AXnWcRforpZSlpZSdgBfqPUtXYLAb3ybAK9X2QmBQHWeRVEcGu/G1AhtX2xtXn0vqggx247sD+HC1vTdwVx1nkVRHBrvxTQUGR8TjwHxqAZfUBfkHZyQpCVfYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhL/BwozvSmyxc9DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import ExtraTreesClassifier # 导入极端随机森林模型\n",
    "ext = ExtraTreesClassifier() # 极端随机森林模型\n",
    "ext.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = ext.predict(X_test) # 进行预测\n",
    "print(\" 极端随机森林测试准确率: {:.2f}%\".format(ext.score(X_test, y_test)*100))\n",
    "print(\" 极端随机森林测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_ext = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AdaBoost 测试准确率: 79.60%\n",
      "AdaBoost 测试F1 分数: 50.72%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAO5UlEQVR4nO3debCV9XnA8e9zAQEBWcTB1kyNCwi1dWmVJWlFUZOpRqz7vqdo1LhF4zKZMdO4zBiNVMXdiHvrhkhNU1c0sggYt0RTTSvuIsSFRUCWX/84L/QGuUTrhXOee7+fmTP3XM573vOc68/vfec9LxqlFCRJja+p3gNIkr4Ygy1JSRhsSUrCYEtSEgZbkpIw2GpYEdHxS2y7UURsXd3vHBFnRMTG1ffDI+LGVfcXEedERK+vMF+viLg0Ijr8f/chfRkGW43smIi4JSLWi4izIuLViHi62e3NiNi/2nYQ8EhEHEZtXc8FHoiIicDPgAtKKUtX7Dgi+gNHVdut+LOnImJGs9uLEbFl9diz1fcfRcR+1VOGA39VSlm2tn8QEsAXPoKR6uDnwPbAhcBbwEWllLErHoyIHwOfAZRSnoqIXYGvARtVt4+BW4GdgbERcVkpZUL19IOprf87I6IJuAEYAHxtRdirI/IVMV4M7AfcBcyKiBeAjYE/RMTTQAAbAH9bSvmstX8QEhhsNbbOpZQTI6IHtaPh8yLiu80e/wtgRkQEsFMp5cmI+B3wE+A54HWgdyllVERsDvQAiIiuwNHA/sC9wF7AB8Ci5i/e/IgcWE7tKP6VUsrTEbE7MB44tnqNya383qXPMdhqSNV54Ycj4s5SytURsR6rP8JuonY0fWlEvAtcDvw9MBjYDngzIvaunrJ+RFwAvAicX0p5LiKWlFJervYHMLX62gMYX0r5YbOxBgDzqvsXUvvFsD2wLWCwtdYZbDWkUsqy6ij2wojYCOgFHFIdYfcCugDvA0+WUj6IiKHAAaWUicDwiDgH6Aqc22y3L5dSPqw+jOwbEScDvSLiFOD5apuhpZSlEXE00H+VscYDt0XEcOA4YMdqlg4RsRPweinl0Nb8OUjNGWw1rFLKwoi4tZQyOyJ2BA4C3qZ2KmMA8Atgy2rbZRHxCkBEfBu4oLptV+3uAOBi4JfUzkt/RC3681jN6ZA1mEDtqHog8Cbwj8B2pZRzvsp7lb4Ig62GFRGdgV9W55+XUfvAbwkwGhgJbA38tNr2b4AbI+Im4EzgJeAfgIXV7rYElgKUUmYDt0TEntSi/Vj1SwH+75QIwG2rjPSXwK7Ag6WUV6vXbd03La2BwVYj25laTD+twrg3sAD4NvCfpZRzI2KT6iqPfYC7gYnAQ8BVwGmllN8DRMRVq9n/EcC/Ag9VV5hAdUqkhXk6AQ8DCyPiWWq/DPoC3SLi74BuwKhSyvSv9ral1TPYamTHAL+oPnAcSO3Su+XULtWjujrkGuBfgCOB3Uopr1WPNQF3RMSKI+z+wLgVO46IXYD+pZSDI+Kn1D5AXK/5i1evG6WUxdRi/UIpZXz18M+rbQ7GUyJaRwy2Gtks4NHq/tPA6FLK8xGxGfBPwBPUzmnPAd5YEevKBsBhpZTfV+e/LwJWxLwncDu1I3aAHwMdqF22N7HZaY4OwKXAfdSCvTqdq5u01oX/AwO1RxHx56WUd+s9h/RlGGxJSsL/logkJWGwJSkJgy1JSay1q0S6bn+yJ8fVkD6avrpLsqXG0KUjLf5tLI+wJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRjsBtGxYxP3jj4egA4dmrjjkmN5/ObTufb8wwBYv8t63H35KB6/+XQuPHXvlc+74Z+P4MlbfsA9o4+nQwf/cWrt+dG5Z3P4IQdyykkn8OmCBZx84vEcsM9IzjvnLEopACxZsoTvn3hCnSdtu9b4b3hE7BYRF0bENRHxk4gYsa4Ga0+6dO7E5DvOZtehAwEYucs2vPjqO4w45nI23mgDthmwCQfvsSPTXprJiGMuZ9AWf8ZWm/XjG9ttTscOTQw/6jI26NaF3YYNrPM7UVv162dnsGzZUm6/624WLFjAuHH30a9fP+4Z9yBz585lyuRJLFq0iEMO3JepUybVe9w2q8VgR8QtwKnAHGAG8BFwRkSMXTejtR+LFi9h8EEX886sjwF4eNLLXHH743To0ESvHuszd8EiPpn3Kd27dqapKejauROfLVnGrA/nMebOiQA0NUX93oDavA379uXQw48CYPny5Vw75iqGDfsmAIMHD2X6tGfo0qUL946bQL9+G9dz1Dat4xoe26aUsv0qf/aziHh+Lc4jYMHCzwB46tYzeX/OJ8x85w+8Pesjzjh6dw7eYwf+41e/5fW356zcfuQu27B8eeHRKb+r18hq4zbd9OsAPPboIzQ1NTFw0CC69+gBQPfu3Zk58/U6Ttd+rOmUyNsRcXVE7BMRI6qv1wBvtfSEiBgVETMiYsbSOb9t/WnbiT49u7Fep47sfPRl9NpgfXbaoT9nHfstbrj3Vwzc83z69FyfodtuBsCew/+aEw/Zmf1OvZZly5bXeXK1ZRMff4w7b7+VK8ZcQ9++fZk/bx4A8+bPo3fv3nWern1YU7D3B56tvv4Q2A+YVn1drVLK9aWUHUopO3Tsu3WrDtqenHrECPbdfXuWLy98uugzunbpRI/1u7Bo8VIAFn+2lG5dO9Nvwx6cfuSu7HvKtcz/dHGdp1ZbNmf2bMbefBNXXn0d3bp1Z/DQYUyeXDtXPf2Zqew4eEidJ2wfWjwlUkpZDNxU3bQOXfdvT3HTBUdywkE78T9vzeaRya/wyn+/z9iLjuKEg3birfc+5Ilp/8XpR+7Kxn17MuHqkwC4ZfwUbh0/tc7Tqy16cPw45syezfdGHQfAd0buzQcfzGL/ffZiwFYDGTJ0WJ0nbB9ixeU4ra3r9ievnR1LX9FH06+q9whSi7p0pMUrCLxwV5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlESUUtbKjhcuYe3sWJLasK6diJYe8whbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpLoWO8B9Md+dN7ZvDHzdfr06cPFl1zG2Weewaz336P/gK244OJLmDF9GmOuHA3Ae+++y0mnnMbIvfep79BqF5qvzctGX0UphR+c9n2uGHMtAIsXL+bM00/5o/UaEXWeum3xCLuBPPfrGSxbtpTb7ryb+fMX8MD999GvXz/uvv9B5s6dy5TJk9hx8BDG3nYXY2+7i/4DtmLgwEH1HlvtwKprc/Kkpzn0wH2ZOmXSym0emvDg59arWpfBbiB9NuzLoYcfBUApy7n26qsY+o1vAjB4yFBmTHtm5bYLFy7krbfeYMBWA+syq9qXVddmRHDPuAn067fxym2mTZva4npV6/CUSAPZdNOvA/D4o48Q0cTAQYPo3r0HAN26dWfmzNdXbjt1yiQGDxlWjzHVDq26NodVYW7uk48/bnG9qnWs8Qg7Ih6MiE8j4tVmt9ci4tUWth8VETMiYsZNN16/diZu4yY+8Rh33nErV4y5hg379mX+/HkAzJ8/j169eq/c7qmJT7DT8F3qNabaoeZrs2PHzx/r9erdq8X1qtbxp06J7Ae8VkoZ0OzWv5QyYHUbl1KuL6XsUErZ4bjvjmr9adu4OXNmc8vNN3HlmOvo1q07Q4YMW3kecNozU9lx8BAASinMmD6NwUOG1nNctSOrrs3VaWm9qvWsMdillCWAP/V1ZML4ccyZPZvvHX8cRx9xCEuXLuWDWbM4YJ+96NmzJ0OG1k6B/OY3L7H5FlvQuXPnOk+s9mLVtfnA/fd+bps9vjNytetVrSdKKWtlxwuXsHZ2LEltWNdOtHgtpFeJSFISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpRElFLqPYO+gIgYVUq5vt5zSKtyba47HmHnMareA0gtcG2uIwZbkpIw2JKUhMHOw3OEalSuzXXEDx0lKQmPsCUpCYPd4CKiS0T8e0S8EBG3RUTUeyapuYjoFBET6j1He2CwG9/hwNullG2B3sDudZ5HWikiugLP4rpcJwx24xsBPFLdfxzYpY6zSH+klLKwlLIN8Ha9Z2kPDHbj2xD4pLo/F+hTx1kk1ZHBbnxzgJ7V/Z7V95LaIYPd+B4DvlXdHwE8UcdZJNWRwW58dwCbRMSLwIfUAi6pHfIvzkhSEh5hS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUn8L+HmskSGofyWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier # 导入AdaBoost 模型\n",
    "dt = DecisionTreeClassifier() # 选择决策树分类器作为AdaBoost 的基准算法\n",
    "ada = AdaBoostClassifier(dt) # AdaBoost 模型\n",
    "ada.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = ada.predict(X_test) # 进行预测\n",
    "print(\"AdaBoost 测试准确率: {:.2f}%\".format(ada.score(X_test, y_test)*100))\n",
    "print(\"AdaBoost 测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_ada = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 梯度提升测试准确率: 86.60%\n",
      " 梯度提升测试F1 分数: 59.64%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPOElEQVR4nO3de5jVdZ3A8fcHmJFLKqCUpWmJkKKLupJru96CLNtSK211QwJJycpIUFuzp7znpiiutuUtFRG6X7zxmC55iaxdJZXWG2qGIZZ5SZAZZrh894/zwyZlWEuYcz4z79fznGfOcH7ndz4zz+95z5ff+Q1EKQVJUuPrVe8BJEmvjcGWpCQMtiQlYbAlKQmDLUlJGGw1rIjo81dsOyQidq7ubxIRUyNiq+rz/SLiilfuLyJOjoiBr2O+gRExLSJ6/637kP4aBluN7KiImBERzRFxUkQsjIh5HW5PRsRh1bY7AbdGxFhqx/VS4McRcTtwAXBWKWXV2h1HxDBgfLXd2j+7MyLu6XBbEBE7VI/Nrz5/ISIOrZ6yH7BLKWX1xv5GSACveQUj1cGVwO7A2cDvgK+UUq5e+2BEnAa0A5RS7oyIMcA2wJDq9ifgGmB/4OqIOL+UckP19COoHf+zI6IXcDkwHNhmbdirFfnaGLcBhwLfAv4QEfcDWwHPRcQ8IIDNgD1KKe0b+hshgcFWY9uklPLpiNiU2mr4lIg4usPj2wL3REQA+5ZS7oiIh4EzgXuBJ4BBpZRJEbE9sClARPQDJgCHAd8HDgKeAVZ0fPGOK3JgDbVV/EOllHkRcQBwHTCxeo27NvDXLr2KwVZDqs4L3xIRs0spX4+IZta9wu5FbTU9LSKWANOBfYA9gd2AJyPikOop/SPiLGABcGop5d6IWFlKebDaH8Avq4+bAteVUj7fYazhwLLq/tnUfjDsDuwKGGxtdAZbDamUsrpaxZ4dEUOAgcC/VivsgUBf4PfAHaWUZyJiL+CjpZTbgf0i4mSgH/CFDrt9sJTyfPVm5JYRcRwwMCImA/dV2+xVSlkVEROAYa8Y6zpgZkTsB3wCeGc1S++I2Bd4opTysQ35fZA6MthqWKWU1oi4ppTyx4h4J3A4sJjaqYzhwBxgh2rb1RHxEEBEvA84q7rtVu3uo8A5wM3Uzku/QC36y1jH6ZD1uIHaqnpH4EngQ8BupZSTX8/XKr0WBlsNKyI2AW6uzj+vpvaG30rgQuBgYGfgvGrbvweuiIhvAicCvwbeD7RWu9sBWAVQSvkjMCMiPkAt2nOrHwrw51MiADNfMdIIYAxwfSllYfW6G/aLltbDYKuR7U8tpi1VGA8BlgPvA35SSvlCRGxdXeXxYeC7wO3ATcDXgONLKY8BRMTX1rH/ccC3gZuqK0ygOiXSyTxNwC1Aa0TMp/bDYEtgQETsDQwAJpVS7n59X7a0bgZbjewoYE71huOO1C69W0PtUj2qq0O+AfwH8HHgPaWUR6vHegGzImLtCnsY8KO1O46IdwPDSilHRMR51N5AbO744tXrRimljVqs7y+lXFc9fGW1zRF4SkRdxGCrkf0B+K/q/jzgwlLKfRHxduAY4DZq57SfBRatjXVlM2BsKeWx6vz3V4C1Md8cuJbaih3gNKA3tcv2bu9wmqM3MA34AbVgr8sm1U3a6ML/wEA9UUS8pZSypN5zSH8Ngy1JSfhviUhSEgZbkpIw2JKUxEa7SqTf7sd5clwN6YW713VJttQY+vah09/GcoUtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmA3iD59evH9Cz8JwB4jtuWxm89k7pVTmHvlFIZt90Z69+7FrHMn8tOrpnDJqWP/4rmTjxzNTZccV4+x1YNdc/VVTPrEBJa++CKfmDCO8WOP4NJv/Ge9x+rW1hvsiHhPRJwdEd+IiDMjYnRXDdaT9N2kibtm/Rtj9toRgIGb9eey781jzMTpjJk4nUcXPcPB7x7JgoVPMfqo6Ww1ZDNGDt8agG3fPIixB+1Zz/HVAy1Z8hQ3XPcjAObcdCNDh+7AjFnf5r57f8Xixb+r83TdV6fBjogZwOeAZ4F7gBeAqRFxddeM1nOsaFvJnoefw1N/+BMAgzbrz4fG7MbPZp7It6YdDcAtP3+Qi679Kb1792Lgpv1ZunwFANNOOowvX3x9vUZXD3XuOWczecoJAJRSWN6ynFIKpcAjDz9c5+m6r/WtsEeWUg4qpUwvpXyzlHJBKeWDwG5dNFuP9fjv/sgZX7+RfcZNY6stN2OfPYaxvLWd1hUrue2qqTzz3FJ++9RzHH7gKBYsfIqHHv99vUdWDzLnxhsY/o4d2X7oUAA+cNDBLFu6lKnHf5bm5ibaVqyo84TdV5/1PLY4Ir4O3Aq8CGwOvBfo9O87ETEJmATQZ5v96bPlzhtw1J5j0ZLneeCxp1++/8bBb2Dw5gN4qaWN/Secz82XTWbfUcN4/7678NatBnHAu3Zi2NvexLGH78sl37mzztOru7vzjtt5+ukl3PXzefz2t0/w7dnXctqZX2Hw4MGcOGUyg7fYot4jdlvrW2EfBsyvPn4eOBT4n+rjOpVSLiuljCqljDLWf7vJR47mowfuQUQwYuibeeDxp/ncuNF85IDdWbOm0LKinX59m5hwytWMmTidcSdfxb0PPWms1SX+/bzzmXHtt/jqtAsYMWJnhg9/B2edfirt7e088vDDjBy5a71H7LY6DXYppa06FTK2lHJgKeXIUspVpZT2rhywJ7rkO3fw8YP34mczT+T62+7n4d/8nku/cyfjD9mL22ecwPN/Ws6tdz1U7zElAP5pn31pb29jwriPMenYT9N/wIB6j9RtRSllo+y43+7HbZwdS6/TC3d/rd4jSJ3q24fo7DGvw5akJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCURpZSNsuNFz7VtnB1Lr1PvXlHvEaRObTOoudMD1BW2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsBvMuWd+kcnHjOXLn/8sq1etYtWqlXzppONefnzZ0qWc+JmJHP/Jj3PtVZfWcVL1RKtWreSLJ9SOx2VLX2Tqp45i8jHjmHnlJQC0trbwpZM+y+RjxnHpxRfUc9RuyWA3kP+9/1esXr2aiy6fRcvy5dzz33fxmaOO4Fd3//LlbW67dQ7bvX0oF156DQ8uuJenlyyu48TqSdpWrODY8Ycz/+5fADD3ljlst/1QLrp8Jg8suI+nlyxm7k9uYqddduWiy2ey6InHWPTEb+o8dfdisBvIoMFb8OF/GQvAmjVriAgunfkDhgx508vblFJobVlOKYVS4PFHH6nXuOphNunblytm/ZAhb6yOx1JobWmpjsXC4wsf4Q1v2JTWlhZWr15NW1sbTU1N9R26mzHYDWTrt27HjiP+jnl3zKVXr17ssee7XrXNmPd9gJdeWsYZp0ylqbmJ9rYVdZhUgvcc+EFeWraM006eQlNTM21tK9h7/zHc/ct5jDv0n9n2bW/nLdu8td5jdivrDXZEXB8RLRGxsMPt0YhY2Mn2kyLinoi4Z/aMKzbOxN3cL352G9d9bzZnnHsxvfv0Wec2U79wOqeeM52mpmYGDtqiiyeU/uzEL57O6V+9kKbmJgYOGszsGVdw0EcOZ/aPf8KypUt5YMF99R6xW/n/VtiHAo+WUoZ3uA0rpQxf18allMtKKaNKKaM+Nv7oDT9tN/f8c8/yvdlXc8Z5F9N/wIB1bvPr++Zz0Xln0t7ezuOPPsJOO4/s4imlmgX3zufCr1bH4sJHGLHLrrS2tNDc3AxAU1MTra0tdZ6ye1lvsEspK4F/6KJZerxb51zP888+yylTPsWUY8dz840/etU273zX3rS3tXPCpyYw9qhJ9Ovfvw6TSrDnP+5Ne3sbxx87niMnfpJ+/ftzyKFHcMMPv8txR4+lva2N3UeZjw0pSikbZceLnmvbODuWXqfevaLeI0id2mZQc6cHqG86SlISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpRElFLqPYNeg4iYVEq5rN5zSK/ksdl1XGHnManeA0id8NjsIgZbkpIw2JKUhMHOw3OEalQem13ENx0lKQlX2JKUhMFucBHRNyJujIj7I2JmRES9Z5I6ioimiLih3nP0BAa78R0JLC6l7AoMAg6o8zzSyyKiHzAfj8suYbAb32jg1ur+T4F313EW6S+UUlpLKSOBxfWepScw2I1vC+DF6v5SYHAdZ5FURwa78T0LbF7d37z6XFIPZLAb31zgvdX90cBtdZxFUh0Z7MY3C9g6IhYAz1MLuKQeyF+ckaQkXGFLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSfwfURXRnncLULUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier # 导入梯度提升模型\n",
    "gb = GradientBoostingClassifier() # 梯度提升模型\n",
    "gb.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = gb.predict(X_test) # 进行预测\n",
    "print(\" 梯度提升测试准确率: {:.2f}%\".format(gb.score(X_test, y_test)*100))\n",
    "print(\" 梯度提升测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_gb = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\jacky.huang\\Anaconda3\\lib\\site-packages\\xgboost\\sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[09:40:30] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.4.0/src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
      "XGB 测试准确率: 85.45%\n",
      "XGB 测试F1 分数: 58.84%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEGCAYAAABB8K+FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAOpElEQVR4nO3de5TWdZ3A8fcHYZCbXEzT1dXVgrytgoLatopC1ra1eUHDC6tlLtmux9Jj3nJbCy9tmdouq5uCoYjXPEaoaYqokaWCoq6ViJp4B1RAZrjI8N0/5oc7KRCtMM/zmXm/znnOPDPP7/k9n5nzO+/5nt/zG4hSCpKk+tep1gNIktaPwZakJAy2JCVhsCUpCYMtSUkYbNWtiOj8Z2y7RUTsWt3vGhGnRsRW1edDI2Lce/cXEWdGRJ8PMF+fiLgoIjb5/+5D+nMYbNWzL0XE1RHREBHfiIjZETG91W1uRBxebbszcHdEHEPLcb0Y+GlE3AdcDJxXSlm5escR0R84rtpu9dceiIgZrW5PRMRHq8dmVp+/FREjqqcMBXYrpTRv7B+EBLDeKxipBq4CBgHnAy8CF5RSJqx+MCLOBVYAlFIeiIjhwLbAFtVtIXANcAAwISJ+UEqZUj39SFqO/+siohNwJTAA2HZ12KsV+eoYLwdGANcDr0fE48BWwBsRMR0IYDNgr1LKig39g5DAYKu+dS2l/HNE9KJlNXx2RJzQ6vHtgBkREcD+pZT7I+L3wBjgMeB5oG8pZXRE7Aj0AoiIbsAXgcOBnwD/AMwDlrV+8dYrcmAVLav435VSpkfEQcBk4PjqNR7cwN+79D4GW3WpOi/8i4i4rpRyWUQ0sOYVdidaVtMXRcQrwCXAfsDewEBgbkQcXD2le0ScBzwB/Fsp5bGIeKeU8ttqfwC/qT72AiaXUk5vNdYA4O3q/vm0/GIYBOwBGGxtdAZbdamU0lytYs+PiC2APsBR1Qq7D7Ap8BpwfyllXkTsCxxRSrkPGBoRZwLdgLNa7fa3pZQ3qzcjPxQRJwF9IuJkYFa1zb6llJUR8UWg/3vGmgxMjIihwJeBIdUsm0TE/sDzpZSjN+TPQWrNYKtulVKWRsQ1pZT5ETEEGAm8RMupjAHAHcBHq22bI+J3ABHxaeC86jaw2t0RwIXAnbScl36Llui/zRpOh6zDFFpW1TsBc4FDgIGllDM/yPcqrQ+DrboVEV2BO6vzz820vOH3DnAp8HlgV+D71bZ7AuMiYjxwGvAk8BlgabW7jwIrAUop84GrI+KztER7avVLAf7vlAjAxPeMtAswHPhZKWV29bob9puW1sFgq54dQEtMm6owHgw0Ap8G7iqlnBUR21RXeRwK3ATcB9wOjAW+XkqZAxARY9ew/38EbgBur64wgeqUyFrm6QL8AlgaETNp+WXwIaBHRPwt0AMYXUp55IN929KaGWzVsy8Bd1RvOO5Ey6V3q2i5VI/q6pDLgR8CxwKfLKU8Uz3WCZgUEatX2P2BW1fvOCIOBPqXUo6MiO/T8gZiQ+sXr143SinLaYn146WUydXDV1XbHImnRNRGDLbq2evAPdX96cClpZRZEbED8E/ANFrOaS8AXlgd68pmwDGllDnV+e8LgNUx7w1cS8uKHeBcYBNaLtu7r9Vpjk2Ai4BbaAn2mnStbtJGF/4HBuqIIuIvSimv1HoO6c9hsCUpCf8tEUlKwmBLUhIGW5KS2GhXiXQbdJInx1WX3npkTZdkS/Vh086s9a+xXGFLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNh1onPnTvzk0q8AsNcu2zHnzjFMveoUpl51Cv2335KuDZ255Ycn8tCNZzJ+zLFrfJ7UFpqamvjaSV/luGOO5JKLvgfAOWedwaijvsDJ/3IiK1eurPGE7dc6gx0Rn4yI8yPi8ogYExHD2mqwjmTTrl14cNIZDN93JwD6bNadK26ezvDjL2H48ZfwzAvzOOqzQ3h53kL2Gfld+mzWneH77vS+50lt4Y7bprD7HgO5etINPPvsHB6dOYPm5pVce/1NNDY28usHf1XrEduttQY7Iq4GvgYsAGYAbwGnRsSEthmt41i2/B32HnkhL7++EIC+m3XnkOED+eXE07j+ohMAOGDIx7j3N78H4P5HZjN0yID3PU9qC70260VTUxPNzc0sW7aMfv36cfSo4wBYtWpVjadr3zqv47HdSymD3vO1iyNi1kacR8CzL87nO5fdxp3Tn2LahFPZb6/+9Ovdg0VLlgKwuHEp/bffssZTqqMaNvwgfjx+HHfcNoX99h/KX+2wIwBT77mbTp068fG/+USNJ2y/1nVK5KWIuCwiDo2IYdXHy4EX1/aEiBgdETMiYsbKBU9t+Gk7iBdeeZN7H3r63ftb9uvJGwuX0LtnNwB69+zGGwsbazmiOrDxV/6IL4w8ip/ffS+LFi1i1mOPct+9U7nu2mv4j/+6nM6d17UO1AexrmAfDsysPp4OjAAerj6uUSnlilLK4FLK4M4f2nWDDtqRnDxqGEf83V5EBLt8ZGueevZVpj389LvnqocOGcD9M2bXeEp1VE1NjTR0bQCgoaGBF+fOZcKPx/Ofl/2IHj161ni69m2tvwpLKcuB8dVNbei/b7yfqy/8El8duT8/m/Y4v3/uNZ57cQEHDxvIwzeexZOzX2ZatQKX2trIo47h7NNP48brr2Orrbdm/vx5LJg/n6+O/jIAhxw2gkMPO7zGU7ZPUUrZKDvuNuikjbNj6QN665GxtR5BWqtNOxNre8zrsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUlEKWWj7PjVRSs2zo4lqR3bundDrO0xV9iSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlETnWg+gP3bhud9k7tw/0LdvP84Z8+9855vfYP7rr7Fj/wGcfe4FLHl7Meec/nWaV65k749/gmO/fGKtR1YH8aeOzWXLlnLev57BooUL2W33QZx48qm1HrndcYVdR56Y9SjNzSu5/KpJNDYu4edTbmWLLT/M+Otu4e3Fi5nx0K+556472GHHjzB23ESefHwWr778Uq3HVgewXsfmnbezy257MHbcRP7w/BxeeP65Wo/d7hjsOtKv3+aMOHIUAGXVKiZceRmD9/k4AHsO3pvHZj5MKYWmxiZKKVAKc2Y/XcuR1UGsz7HZs2cvljY10dzczPJly+ncpUstR26XDHYd2Xa77dl517/ml9OmEp060X/AzvTo2ROA7j168vbiRRz0mc+xZMnbfOuMU+jS0MDy5ctqPLU6gvU5Nvc7cDgP/2Y6Rx/292y/ww5ss+1f1njq9medwY6In0VEU0TMbnV7JiJmr2X70RExIyJmXDth3MaZuJ371QPTuOWmSVz4g7H023xzGpcsAaCxcQm9e/cF4PRzvs2Y711Kly5d6NOvXy3HVQfyp47NSRPG8fnDRnLj5LtYvHgx//PErNoO3A79qRX2COCZUsqAVrf+pZQBa9q4lHJFKWVwKWXwqC+esOGnbefeWLCAGyZO4MKLx9K9Rw/2HLIvMx56EIDHZjzEwMFDeOKxmVz83TGsWLGCOc88zS677VHjqdURrM+x2dTUREPXBgC6dOnC0qamWo7cLq3zKpFSyjsRsU9bDdPR3XX7ZN54Yz7fOLnlyo9PfeZzzJ83j+OPPoyP9P8Yew3Zl+bmlUy59WZOHn0cxx7/Fbp3717jqdURrM+xuc2223H+t87kpzffwIe32po9h5iODS1KKRtlx68uWrFxdixJ7djWvRtibY/5pqMkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrCYEtSEgZbkpIw2JKUhMGWpCQMtiQlYbAlKQmDLUlJGGxJSsJgS1ISBluSkjDYkpSEwZakJAy2JCVhsCUpCYMtSUkYbElKwmBLUhIGW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSUQppdYzaD1ExOhSyhW1nkN6L4/NtuMKO4/RtR5AWguPzTZisCUpCYMtSUkY7Dw8R6h65bHZRnzTUZKScIUtSUkY7DoXEZtGxG0R8XhETIyIqPVMUmsR0SUiptR6jo7AYNe/UcBLpZQ9gL7AQTWeR3pXRHQDZuJx2SYMdv0bBtxd3b8XOLCGs0h/pJSytJSyO/BSrWfpCAx2/dscWFTdXwz0q+EskmrIYNe/BUDv6n7v6nNJHZDBrn9TgU9V94cB02o4i6QaMtj1bxKwTUQ8AbxJS8AldUD+4YwkJeEKW5KSMNiSlITBlqQkDLYkJWGwJSkJgy1JSRhsSUrifwG6jc1LVvISgQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from xgboost import XGBClassifier # 导入XGB 模型\n",
    "xgb = XGBClassifier() # XGB 模型\n",
    "xgb.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = xgb.predict(X_test) # 进行预测\n",
    "print(\"XGB 测试准确率: {:.2f}%\".format(xgb.score(X_test, y_test)*100))\n",
    "print(\"XGB 测试F1 分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "show_matrix(y_test, y_pred)\n",
    "f1_xgb = f1_score(y_test, y_pred)*100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from xgboost import XGBClassifier # 导入XGB 模型\n",
    "# xgb = XGBClassifier() # XGB 模型\n",
    "# # 使用网格搜索优化参数\n",
    "# xgb_param_grid = {'min_child_weight': [1, 5, 10],\n",
    "# 'gamma': [0.5, 1, 1.5, 2, 5],\n",
    "# 'subsample': [0.6, 0.8, 1.0],\n",
    "# 'colsample_bytree': [0.6, 0.8, 1.0],\n",
    "# 'max_depth': [3, 4, 5]}\n",
    "# xgb_gs = GridSearchCV(xgb, param_grid = xgb_param_grid,\n",
    "# scoring=\"f1\", n_jobs= 10, verbose = 1)\n",
    "# xgb_gs.fit(X_train, y_train) # 拟合模型\n",
    "# xgb_gs = xgb_gs.best_estimator_ # 最佳模型\n",
    "# y_pred = xgb_gs.predict(X_test) # 进行预测\n",
    "# print(\"XGB 测试准确率: {:.2f}%\".format(xgb_gs.score(X_test, y_test)*100))\n",
    "# print(\"XGB 测试F1分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))\n",
    "# show_matrix(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Voting 测试准确率: {:.2f}% 86.0\n",
      "Voting 测试F1分数: {:.2f}% 59.06432748538012\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import VotingClassifier # 导入Voting 模型\n",
    "# 把各种模型的预测结果进行Voting。同学们还可以加入更多模型如SVM, KNN 等\n",
    "voting = VotingClassifier(estimators=[('rf', rf),('bdt', bdt),('gb', gb),\n",
    "                                      ('ext', ext),('xgb', xgb),('ada', ada)],\n",
    "voting='soft', n_jobs=10)\n",
    "voting = voting.fit(X_train, y_train) # 拟合模型\n",
    "y_pred = voting.predict(X_test) # 进行预测\n",
    "print(\"Voting 测试准确率: {:.2f}%\", voting.score(X_test, y_test)*100)\n",
    "print(\"Voting 测试F1分数: {:.2f}%\", f1_score(y_test, y_pred)*100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import GridSearchCV # 导入网格搜索工具\n",
    "# from sklearn.ensemble import RandomForestClassifier # 导入随机森林模型\n",
    "# rf = RandomForestClassifier() # 随机森林模型\n",
    "# # 使用网格搜索优化参数\n",
    "# rf_param_grid = {\"max_depth\": [None],\n",
    "# \"max_features\": [1, 3, 10],\n",
    "# \"min_samples_split\": [2, 3, 10],\n",
    "# \"min_samples_leaf\": [1, 3, 10],\n",
    "# \"bootstrap\": [True, False],\n",
    "# \"n_estimators\" :[100, 300],\n",
    "# \"criterion\": [\"gini\"]}\n",
    "# rf_gs = GridSearchCV(rf, param_grid = rf_param_grid,\n",
    "# scoring=\"f1\", n_jobs= 10, verbose = 1)\n",
    "# rf_gs.fit(X_train, y_train) # 拟合模型\n",
    "# rf_gs = rf_gs.best_estimator_ # 最佳模型\n",
    "# y_pred = rf_gs.predict(X_test) # 进行预测\n",
    "# print(\"调优后随机森林测试准确率: {:.2f}%\".format(rf_gs.score(X_test, y_test)*100))\n",
    "# print(\"调优后随机森林测试F1分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = np.asarray(X_train).astype(np.float32)\n",
    "X_test = np.asarray(X_test).astype(np.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 12)                144       \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 24)                312       \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 48)                1200      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 49        \n",
      "=================================================================\n",
      "Total params: 1,705\n",
      "Trainable params: 1,705\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/30\n",
      "100/100 [==============================] - 1s 5ms/step - loss: 0.5404 - acc: 0.7692 - val_loss: 0.4967 - val_acc: 0.7875\n",
      "Epoch 2/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4716 - acc: 0.8000 - val_loss: 0.4842 - val_acc: 0.7875\n",
      "Epoch 3/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4601 - acc: 0.8009 - val_loss: 0.4675 - val_acc: 0.7950\n",
      "Epoch 4/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4511 - acc: 0.8069 - val_loss: 0.4558 - val_acc: 0.8012\n",
      "Epoch 5/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.4402 - acc: 0.8133 - val_loss: 0.4456 - val_acc: 0.8056\n",
      "Epoch 6/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4315 - acc: 0.8142 - val_loss: 0.4341 - val_acc: 0.8169\n",
      "Epoch 7/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4244 - acc: 0.8161 - val_loss: 0.4270 - val_acc: 0.8175\n",
      "Epoch 8/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4225 - acc: 0.8213 - val_loss: 0.4234 - val_acc: 0.8169\n",
      "Epoch 9/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.4144 - acc: 0.8197 - val_loss: 0.4172 - val_acc: 0.8194\n",
      "Epoch 10/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4081 - acc: 0.8253 - val_loss: 0.4088 - val_acc: 0.8269\n",
      "Epoch 11/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3992 - acc: 0.8248 - val_loss: 0.3985 - val_acc: 0.8300\n",
      "Epoch 12/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3859 - acc: 0.8350 - val_loss: 0.3860 - val_acc: 0.8344\n",
      "Epoch 13/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3755 - acc: 0.8417 - val_loss: 0.3753 - val_acc: 0.8419\n",
      "Epoch 14/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3651 - acc: 0.8491 - val_loss: 0.3705 - val_acc: 0.8475\n",
      "Epoch 15/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3601 - acc: 0.8477 - val_loss: 0.3636 - val_acc: 0.8469\n",
      "Epoch 16/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3548 - acc: 0.8525 - val_loss: 0.3586 - val_acc: 0.8544\n",
      "Epoch 17/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3540 - acc: 0.8514 - val_loss: 0.3559 - val_acc: 0.8531\n",
      "Epoch 18/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3501 - acc: 0.8533 - val_loss: 0.3555 - val_acc: 0.8587\n",
      "Epoch 19/30\n",
      "100/100 [==============================] - 0s 4ms/step - loss: 0.3495 - acc: 0.8547 - val_loss: 0.3558 - val_acc: 0.8512\n",
      "Epoch 20/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3480 - acc: 0.8561 - val_loss: 0.3517 - val_acc: 0.8575\n",
      "Epoch 21/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3451 - acc: 0.8573 - val_loss: 0.3524 - val_acc: 0.8550\n",
      "Epoch 22/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3446 - acc: 0.8591 - val_loss: 0.3525 - val_acc: 0.8544\n",
      "Epoch 23/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3440 - acc: 0.8584 - val_loss: 0.3485 - val_acc: 0.8575\n",
      "Epoch 24/30\n",
      "100/100 [==============================] - 0s 3ms/step - loss: 0.3414 - acc: 0.8592 - val_loss: 0.3528 - val_acc: 0.8512\n",
      "Epoch 25/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3422 - acc: 0.8597 - val_loss: 0.3496 - val_acc: 0.8544\n",
      "Epoch 26/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3411 - acc: 0.8592 - val_loss: 0.3448 - val_acc: 0.8606\n",
      "Epoch 27/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3400 - acc: 0.8608 - val_loss: 0.3450 - val_acc: 0.8581\n",
      "Epoch 28/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3387 - acc: 0.8616 - val_loss: 0.3440 - val_acc: 0.8594\n",
      "Epoch 29/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3383 - acc: 0.8627 - val_loss: 0.3436 - val_acc: 0.8587\n",
      "Epoch 30/30\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3384 - acc: 0.8617 - val_loss: 0.3407 - val_acc: 0.8587\n"
     ]
    }
   ],
   "source": [
    "import keras # 导入Keras库\n",
    "from keras.models import Sequential # 导入Keras序贯模型\n",
    "from keras.layers import Dense # 导入Keras密集连接层\n",
    "ann = Sequential() # 创建一个序贯ANN模型\n",
    "ann.add(Dense(units=12, input_dim=11, activation = 'relu')) # 添加输入层\n",
    "ann.add(Dense(units=24, activation = 'relu')) # 添加隐层\n",
    "ann.add(Dense(units=48, activation = 'relu')) # 添加隐层\n",
    "ann.add(Dense(units=1, activation = 'sigmoid')) # 添加输出层\n",
    "ann.summary() # 显示网络模型（这个语句不是必须的）\n",
    "# 编译神经网络，指定优化器，损失函数，以及评估标准\n",
    "ann.compile(optimizer = 'Adam', loss = 'binary_crossentropy', metrics = ['acc'])\n",
    "history = ann.fit(X_train, y_train, # 指定训练集\n",
    "                  epochs=30,        # 指定训练的轮次\n",
    "                  batch_size=64,    # 指定数据批量\n",
    "                  validation_split=0.2) #指定验证集,这里为了简化模型，直接用训练集数据\n",
    "# show_history(history) # 调用这个函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "神经网络测试F1分数: 57.80%\n"
     ]
    }
   ],
   "source": [
    "y_pred = ann.predict(X_test,batch_size=10) # 预测测试集的标签\n",
    "y_pred = np.round(y_pred) # 将分类概率值转换成0/1整数值\n",
    "print(\"神经网络测试F1分数: {:.2f}%\".format(f1_score(y_test, y_pred)*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6kAAAE8CAYAAAAi6/dqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeX0lEQVR4nO3de7xnZV0v8M/XBsTrcHFMRGnIC540Uc9oqJSAgmZY5iUvedcm1KzUzMxX5+gxNTNDPXo0PF7xVqYWYpDKpdIUHS+QWmoEJh6zQRA0UEG+54+1tmy2AzMjs/Zem/1+v17z2uu31vqt5/nNfvbv9/us53nWqu4OAAAAzMF1VroCAAAAsEBIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNtatdAW25SY3uUlv3LhxpasBAADABD75yU+e190btrVtliF148aN2bJly0pXAwAAgAlU1ZevapvhvgAAAMyGkAoAAMBsCKkAAADMhpAKAADAbAipAAAAzIaQCgAAwGwIqQAAAMyGkAoAAMBsCKkAAADMhpAKAADAbAipAAAAzMa6qQ5cVb+b5EFJLkjysCRvT3LLJGcmeUx391Rlw650zgEHrHQVmMjGs89e6SoAALDEJD2pVfWTSW7f3QcnOTHJw5Oc290HJdkryRFTlAsAAMDqNlVP6r2T7FVVf5/k60kuS/KX47ZTkhyW5AMTlQ0AAMAqNdWc1A1Jtnb3zyW5RZKbJrlw3HZRkr0nKhcAAIBVbKqQelGSL4zL/5bk0CTrx8frk5y39AlVtbmqtlTVlq1bt05ULQAAAOZsquG+n0zy9HH51kmek+TIJO9OcniSY5Y+obuPTXJskmzatMlFlQAAVsBeT99rpavARC445oKVrgLskEl6Urv7o0m+UVWfyNCj+ook+1XVmUnOT3LyFOUCAACwuk12C5rufvKSVUdNVRYAAADXDlPNSQUAAICdNllPKgAAwAEHnLPSVWAiZ5+9cZLj6kkFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZmPdSldgVzvn9QesdBWYyMYnnr3SVQAAACamJxUAAIDZEFIBAACYDSEVAACA2RBSAQAAmA0hFQAAgNkQUgEAAJiNa90taADm7ul7PX2lq8BEjrngmJWuAgCsenpSAQAAmA0hFQAAgNkQUgEAAJgNIRUAAIDZEFIBAACYDSEVAACA2RBSAQAAmA0hFQAAgNkQUgEAAJgNIRUAAIDZEFIBAACYDSEVAACA2RBSAQAAmA0hFQAAgNkQUgEAAJgNIRUAAIDZEFIBAACYjXVTHLSq7prkvUnOGVc9NckLk9wyyZlJHtPdPUXZALDWnHPAOStdBSay8eyNK10FgGU3VU/qXkle092HdPchSe6a5NzuPmjcdsRE5QIAALCKTdKTmiGIPriqfinJV5J8L8lfjttOSXJYkg9MVDYAAACr1FQ9qf+a5A+6+25J9k3yoCQXjtsuSrL30idU1eaq2lJVW7Zu3TpRtQAAAJizqULqOUk+tGj58iTrx8frk5y39AndfWx3b+ruTRs2bJioWgAAAMzZVCH1GUkeXlXXSXKHJM9McuS47fAkp05ULgAAAKvYVCH1VUken+T0DFf5fX2S/arqzCTnJzl5onIBAABYxSa5cFJ3fy3JoUtWHzVFWQAAAFx7TNWTCgAAADtNSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNiYNqVX1jKr6UFXtUVUnVNUZVXVcVdWU5QIAALA6TRZSq+onkjx2fPioJOd290FJ9kpyxFTlAgAAsHpN2ZP6iiTPGZcPT/LBcfmUJIdNWC4AAACr1CQhtaoemeSMJJ8fV+2T5MJx+aIke09RLgAAAKvbuomOe1SS/ZPcN8mBSS5Psn7ctj7JeUufUFWbk2xOkv3333+iagEAADBnk/Skdvcju/uQJA9P8skkz0py5Lj58CSnbuM5x3b3pu7etGHDhimqBQAAwMwt1y1o3pZkv6o6M8n5SU5epnIBAABYRaYa7psk6e5zktxnfHjUlGUBAACw+i1XTyoAAABsl5AKAADAbAipAAAAzIaQCgAAwGwIqQAAAMyGkAoAAMBsCKkAAADMhpAKAADAbAipAAAAzIaQCgAAwGwIqQAAAMyGkAoAAMBsCKkAAADMhpAKAADAbAipAAAAzIaQCgAAwGwIqQAAAMzGdkNqVf14VW26im133fVVAgAAYK1atwP73DLJW6rqY0m+muTTSU5McniS309yz+mqBwAAwFpytSG1qvZNcnmSdyR5TYbA+sBx+StJ7jNx/QAAAFhDtteT+kdJfjrJBUnOS3KHJLsneViSpyY5OMlJU1YQAACAtWN7IfW3klyU5DlJbp/knO5+aZJU1VlJ3ldVp3X3d6atJgAAAGvB9kLqryW5OMlhSS5J8rqqOj7Ju5I8JcnjBFQAAAB2le1d3XfPJDdNcoskX0hSSW6QZH2S3ZKcMWXlAAAAWFu2F1JPTPKxJPtmCKsPSHJgkjsl+YskvzNl5QAAAFhbthdSD0ny/SSnJ/lckk8l+fckn+nuP07ys1X1Y9NWEQAAgLXiauekdvcfJUlV/WuSLyfZJ8lx3f2acZcXZhgCDAAAANfY9npSkyTd/W9JbpXkkkUBNd19endfNlXlAAAAWFt2KKRW1e5J3pDk8dNWBwAAgLVse7egSVXdNMlxGeal3q6qfi/JV5Kck+Tj3X3ppDUEAABgzbjantSqek6Sk5K8vrufmeTgJP+SZK8km5O8dvIaAgAAsGZsryf1r5O8pLsvHx9/s7v/KkmqakOS50xYNwAAANaY7c1JvV6Smy96fEBVPb6qNnb31u5+xoR1AwAAYI3ZXkjdkOSEqnrs+PiiDPdNfV1V/a9JawYAAMCac7UhtbtPSnKPJD9bVW9K8rrufkt3H5HkFlW1eRnqCAAAwBqx3VvQdPfF3f2kDL2od1i06elJ7rat51TVuqp6V1V9pKreUFV7VNUJVXVGVR1XVbVrqg8AAMC1yQ7dJ3X0jCQvXXjQ3ReO4XVbHpjkjO6+Z5J9k/xGknO7+6AMVwY+4kerLgAAANdmOxxSu/uy7j5rB3c/KcmfVtW6JHsmuUuSD47bTkly2M5UEgAAgLVhZ3pSd1h3f7u7L07ykSRfT7JPkgvHzRcl2Xvpc6pqc1VtqaotW7dunaJaAAAAzNx2Q2pVHV9VF1fVFxf9+1JVffFqnrNPVV03w0WX9sowl3X9uHl9kvOWPqe7j+3uTd29acOGDT/aqwEAAGBV25Ge1Acn+VJ333bRv9t0922v5jnPTPLQ7v5+kouTvDDJkeO2w5Oceo1qDQAAwLXSjoTUX0/yMzt53FcneUJVfTTJN5K8Psl+VXVmkvOTnLyTxwMAAGANWLcD+/xKd78qSarqVd39G9t7Qnd/NUOP6WJH/Qj1AwAAYA3Z2Qsn/dQktQAAAIDsWE/qTavqkUkqyc3G5SRJd799spoBAACw5uxISH1HktuMy3++aLknqREAAABr1nZDanc/fzkqAgAAADs7JxUAAAAmI6QCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbk4XUqnpzVX2sqo6vqhtW1QlVdUZVHVdVNVW5AAAArF6ThNSqOiTJuu4+OMmNkzwhybndfVCSvZIcMUW5AAAArG5T9aR+PckrFpXxvCQfHB+fkuSwicoFAABgFVs3xUG7+0tJUlW/nOTyJJ9OcuG4+aIkB05RLgAAAKvblHNSfzHJbyZ5QJL/SLJ+3LQ+yXnb2H9zVW2pqi1bt26dqloAAADM2FRzUm+W5FlJjurubyU5OcmR4+bDk5y69DndfWx3b+ruTRs2bJiiWgAAAMzcVD2pj02yb5K/raoPJ9ktyX5VdWaS8zOEVgAAALiSqeakviTJS5as/rMpygIAAODaY7I5qQAAALCzhFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNoRUAAAAZkNIBQAAYDYmC6lVtVtVvW9c3qOqTqiqM6rquKqqqcoFAABg9ZokpFbV9ZJ8MskR46pHJTm3uw9Kstei9QAAAPADk4TU7r6ku++Y5Nxx1eFJPjgun5LksCnKBQAAYHVbrjmp+yS5cFy+KMneS3eoqs1VtaWqtmzdunWZqgUAAMCcLFdIPS/J+nF5/fj4Srr72O7e1N2bNmzYsEzVAgAAYE6WK6SenOTIcfnwJKcuU7kAAACsIssVUt+WZL+qOjPJ+RlCKwAAAFzJuikP3t23Hn9+N8lRU5YFAADA6rdcPakAAACwXUIqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbQioAAACzIaQCAAAwG0IqAAAAsyGkAgAAMBtCKgAAALMhpAIAADAbyxZSq2qPqjqhqs6oquOqqparbAAAAFaH5exJfVSSc7v7oCR7JTliGcsGAABgFVjOkHp4kg+Oy6ckOWwZywYAAGAVWLeMZe2T5MJx+aIkBy7eWFWbk2weH367qr6wjHVbzW6S5LyVrsSyeJIR4stk7bQpsw6Ww9ppT0leXi9f6SqsBWuqTcXb1HJYM22qXq5BLZO106auWZP6iavasJwh9bwk68fl9Vnyi+vuY5Mcu4z1uVaoqi3dvWml68G1hzbFrqQ9satpU+xq2hS7mjZ1zS3ncN+Tkxw5Lh+e5NRlLBsAAIBVYDlD6tuS7FdVZyY5P0NoBQAAgB9YtuG+3f3dJEctV3lriCHS7GraFLuS9sSupk2xq2lT7Gra1DVU3b3SdQAAAIAkyzvcFwAAAK6WkDqRqnpEVT1jG+vvWVUvWvT4t6rqLoseP6Oq9liuenLtVVV3W9y2lqnM6y9neVwzVbX7BMdc9nY3lqvtzVhV3aiqvl1VN1qy/nlVdehVPOe0qvp4Vf1jVb1yF9Xj0KrauCuOxfzV4JVV9dGqek9VvaiqvlBV/1BVJ1XVjavqcVX15araUlUvr6rdquqhVfXhqrpk/PnUlX4tLI+qenRVvWxcfnNV/fySNrT7+L51pXa0C8rdeFXvhWuVkDqBqrpOksuSfHN8o3tbVf3luPmyJJeO+yTJ/ZN8fXzefkke2t3fWXSsXf6HMB73f++K47DyqurOVXVOVX2pqp5YVTcdNz0+S+6wV1WfWLR8wfgl8IKquu6i9T9WNdz1qqqeUFW/tnjbouUbVdVHllTnY7vwpTGRqvr7cfGEqvpGVX1obEMPGbf/wsJJtqp63/jz+PHn+8efO9zuxv21vbXt8CTXTXLYTj7vQd19jyQHVtXtdkE9Dk2ycRcch9Xh3kkO6O67J/lckocmeUF3/2ySv0vyyHG/1yU5OMntkmzu7nd19yFJvtrdh3T3q1eg7qyMtye5V1UdmOSAJJfmh9tQsu12dE1szPD+xGg575O6lvxikhdnCKRHd/evVtVxVfWGJD+d5MeTnF5VL07yrSTvrqqzknw+yV5V9eEk+yVZ6I14QXe/taqek+EP4bXXtILd/bRregxm47tJ3pTkxkm+muRZVXWbJPdKcsuxt+y5ST6V5MKqWpekk3ymuw+rqg8l+d6i470wyV2rap+MX+aq6ilJvpHk35JsXnQiZuGES/cwwf3icf9147rvT/rK2SlVtSHJhiSXV9Udkvx8kpO6+4iqenuSM8ZdP5mhXSXD73jdop8LJ9F2pN09O8k/ZWhv2t7adr8kr05yv/EkybuS7J7k+0lOq6p9k/xFhu8lp3T3cxeeOP5O1yf5TlXtneQtSfZJcnp3//ZVrNswHu96ST7d3U+uqrdk+BL44Kr6XHc/fFleOSvp0Fxxy8NXZvj+tWBDkv+X8aRad182tpFfztBWWYO6+/tjR85JSZ6aH25DuyW5zaKnLLSjhQ6gTRk+sx7d3RcsXZfkmxk+O2+T4XPrQUmemOQJSdZX1X2S/HJ3b53sRa4SelIn0N1/leSDSf4owxepozOE0ZckeXKSN3T33yT5ynim7qgklyc5JMlB47rPJ7lwyaE3JLlk7G14a1WdXlXH1zA05XpjT+uWMRA/d1vrFg5UVactWn5cVb2sqv6uqv65qm5fg7dX1Seq6i+q6pip/r+4xi7P8MVuzwzt7ttJzsrwJfCtSf44wwfzh5P8TJIPZThb/INw0IuuoNbdv5fkSeMxnp8hOJyT5NndvXnc7b4Z3rTvmOSUJI+uqlOT/Mz48wNJDpzixXKN3C3DCbQDMwTIPZPsPW47IMlZVXVkkvcmeV5VbU7yxSTvzBAu35nk7HH/7ba77v50hnCi7XFokhdkOImxOcnfdPe9MrSjJLllhpNpR2Y40bvgXRnaw1eSfDnJ7yf587FXY6+quu9VrPu5JJ/t7oOTfKSqrtPdj0nyhiRPE1DXjA1JLqqqRyd5X4ZA8NyqOiNDO/vrJfufl2Sv5a0iM3Rykptn+OzaVhtKlrSjqjoqyfXH96F3J/ndba3L8Jl7pyT3TPKHSfbs7mOSPC1DPjhEQB0IqRMYh6vdO8mvJbl+koclOT3DmbnFvdf7VtWWDF/cvp/k6CTPG7ddp7sXPryXvqHuk+GL2CFJLsrQ43q7DB/id09y2+5+4VWsuyp3T3JEhmD9Sxm+eN6ku++a5Nbd/fQf5f+CZXPDJHtkaEP7J9ma4Xe/JcMXshMy/G7PSnLv7v5chtDxQ6rqTzIE26ckuSDDfY2PTnJ0Vf1tVe3e3ScmOSbJmUke1t1vSbIwhPy13X14d39+mpfKj6q735/hg/IL3f3oDL1Md6mq3TL8ns9P8s9Jfj3JyzK0n/MzDKVd+PefVXXz8ZBX2+7GMrW9Na6qbpvkZhna3s0znGhY6LVfGAb+3STPTPJnGdrVgocmuXWSH0vyqCQ/leSj47aPjo+3te7EsewTktxq0ecpa8uFSW7U3cdl+H61Z5IXdvdBSf5Phve5xfbO8N7D2vbMJO/J8Dm2rTaU/HA72qH3pu7+Roae1OMz9KxeNO1LWb2E1Gk8KMPwt3dmGBbw3Qzzcd6eYWjcgq9196Yk90mS7v5ykkPqhy8AsvQP4XsZzjb/eZKfyPBF86sZwurfZRiOkKtYd1Xe3t3fy3CmevcklyTZo6o+muRtO/PiWRHfzDDc5JQkN8gQEu6S5CG54sTIwzIEir+pqutlODFyJeMJlpdm6G24Z5J/zHAm8WkZzvjdP8P8jCT51ST7Jnn/GHKOTvKlJI+vCS7Iwy7zsCQ3qKrXJ/nJDKHy/hlOTp2eYd7gE5M8IMN7yBuSvDlDe/q/SV6fIYwmO9buFsrU9tau+yZ5aXcfmuRPMnwm3WHctjCt5XcynKDYnCUnMbr70gyjkW6YYU7YweOmg8fH21p3zyTv6O6jkhxZVbcat1+Soa0utDmu3T6S5P7j7/qgJdsuyKITIjXMe39UhpEhrFFV9eMZOoEem+Gz7OraUHJFO9qh96aq2j/JN7v7AUn+I8mDx+3em5YwJ3UaF2cIpJdmmEfzqfHnJzJ8Abw6j+3ui6+ifS78ITw4Q2/HizIE1WQYUveH3f3eRftva91V+faSx3dL8t5xCAKrw5kZhtItuH2GYUuXVtUtMgxZOTfD3K2fSvKf2zjGPTIMB73zuO8fJ3lMhqF6T0ryL0l+Z3wT/1aSr2UYwvfYDG/GN84wVPTVVbV58VBOVl5VPTBDj2ZnGM50jyS/nSGUXpbhy9mXMwz9vTjDh+YXx6d/I0NP6l4Zwu2Cq2x3Y5naHvfNcKIhGU5oPDfJDavqVxbt874MF6/5apL/WtRb/55x7vHWDCdM1yU5roarrZ7e3R8YRyQtXbcxyVtruDDXuRnadTL05r6pqv5Hhms8nDXNS2Ymjs/wXejjuWIK1R+MbWVdhhFvdxp//lKGkW2vX/5qMiO/m+RV3f29qnpThhNq/5or2tBp435Xakfd/Zmqut/YubN4TuqV1mX4vn3/cSpg5YoLMX06w8jJj2ToWFr4fr9mlc/xadRwlcz1GXpJn5Tke919aVVtSnJUdz+vqj6VK+Z1nTnOl8nYM3B8d/98VT0vySMyDHtbeEO9JMMH7YVJ/ivDsIF/yjAE+IsZvgA+P0Pjv9K67v7sWMZp41ntVNXjkqS731TD5a8PzdBr+5kMf5jnJ3lZd//Drv5/4pqr4YqXT8wQDp6YYSjkGzMEjY9nOJnxVxm+jD27u+9TVY9KsqG7j6mqD3X3fRYd745JTkjy/iSfzdDr9YwkX0jyhO7eWlV3yxBiXpVhaOaJGS4M8L7uPriGi0+c2t1vnPw/gB1WVYdkCHtv6u6jxg/Oe2d4j/mfSe7c3d+o4Uq9b0zynAy/+2RoEzfMFe9pG7Oddtfd9xs/iLU9AGCH6Umdzm5JduvuRyysGIehvSbDBZSS5MTufm5V3TDJ48Z97pdh/tbLkqS7n5cr5qkudofFD2q4VcNZGXpD9khykwxXDlu6LuNxD120/KZFy6dluNLiL2Q48/z9DMPC99uJ187yWpfkv7r7/lX133LFUMktGU5kPDvJ2d19UQ0X1FqX4QJevzo+v6vqv2e4cMkZGeZOvzjDyZCvZRh+98Ek70jylqp6end/fBy2WRlOZPziGCAW6vT4XHFBFGaiuz+c/OBWL4/MECZvmmGI25cyXD33nAy/v8/lipEfu2VoZ4uniGy33VXVjbv7tWOZ2h4AsEOE1Il09zu2se57VXX37r5sfPzc8ee3M/QKpLtPynDZ650t73UZhkotdto21u3o8d6foTeD+dtt0fKDMvSE3y3DMMj9M8xxeGaG4eY3yjA0c0t3nzM+Z0uSVyR5Vob5iY/OMB/jehkudrI5Q8/81zL0oC3cgmS3JNcd5zIvXATl+slwCfdd+xLZxfbJMOz2rRmGUD41w7DdF2UYynvf7u6q2nPc//kZLu5w5yS3yjDPfmfaXaLtAQA7yHBfWIOqqszZY2eNUxFqDIc/6jG0PQDgagmpAAAAzIZb0AAAADAbQioALIOquvWi5euMt0kBAJYQUgFgYlX12CTvraqjq+pJGa7ofuL4+ClVtVtVPW28r+zCc/avqvesVJ0BYKUIqQAwoTF4PjTDlYp/LsPtvR6a4dY/RyU5PcPtvr6S5OSq2me8SNV3k1w6HsPV+AFYM1w4CQCWQVW9N8Ptfxbfx/Wb3f3ARfvcKclXk7wrw71gD0zy2SSXdPcDlq2yALCCnJkFgOVxyyTvXLLuEUlSVfdI8uQkr+jurUkOraqbJHlVdz98easJACtLTyoALIOq+nyS/1yy+mbdfbtx++2T3DjDUODbJVmfodf1wiS7J3lld5+wfDUGgJWhJxUAlsfpGeadXjj+u2OG+alJku7+XFU9JMlzu/uSqjouyRuTnNrOKAOwhrhwEgBMqKruVVWnJdk3w8WSHjL+e0SSm1XVKVV1v6o6IMkTknynqm6T5FtJ/j3Ju1em5gCwMgz3BYBlMF4U6dVJ7tXdl1XVh5L8Qnd/d7x67weS/GaS/0jy5iQvSLJnkhcneUl3L53PCgDXSnpSAWBiVbU5yZ8meXR3Xzau3i3DXNMkuVWSj3T3Z5PcIMklSe6R4Qq/Ryc5dFkrDAArSE8qAEysqq7T3Zdvf08AQEgFAABgNgz3BQAAYDaEVAAAAGZDSAUAAGA2hFQAAABmQ0gFAABgNv4/u+/Ct/9tm+MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1152x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plt.rcParams['font.sans-serif']=['SimHei'] #用来正常显示中文标签\n",
    "# plt.rcParams['axes.unicode_minus']=False #用来正常显示负号\n",
    "# plt.rcParams[\"font.family\"]=['SimHei']\n",
    "methods = [\"树的Bagging\", \"随机森林\", \"极端随机森林\",\n",
    "\"AdaBoost\", \"GBDT\", \"XGBoost\"]\n",
    "f1 = [f1_bdt, f1_rf, f1_ext, f1_ada, f1_gb, f1_xgb]\n",
    "colors = [\"orange\", \"red\", \"purple\", \"magenta\", \"green\", \"blue\"]\n",
    "# sns.set_style(\"whitegrid\")\n",
    "plt.figure(figsize=(16, 5))\n",
    "plt.yticks(np.arange(0, 100, 10))\n",
    "plt.ylabel(\"F1分数\")\n",
    "plt.xlabel(\"算法\")\n",
    "sns.barplot(x=methods, y=f1, palette=colors)\n",
    "# plt.grid(b=None)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
